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ABSTRACT 


This  thesis  analyzes  the  effect  of  quantitative  and  qualitative  factors  used  in  the 
admissions  process  at  the  Naval  Academy  in  determining  student  performance  of 
candidates  admitted.  In  determining  student  performance,  graduation,  Order  of  Merit, 
cumulative  academic  QPR,  cumulative  military  QPR,  and  striper  selection  are  used  as 
performance  outcome  measures.  The  data  is  from  Naval  Academy  graduation  year 
groups  1995  through  2001.  The  analysis  separates  the  Naval  Academy’s  Whole  Person 
Multiple  into  quantitative  and  qualitative  inputs.  The  Candidate  Multiple  (CM)  is  the 
quantitative  input  to  the  admissions  process  derived  from  a  statistical  based  scoring 
model  anchored  in  proven  high  school  performance  measures  such  as  SAT  and  high 
school  GPA.  The  Recommendations  of  the  Admissions  Board  (RAB)  is  the  qualitative 
input  awarding  points  for  subjective  traits  not  captured  in  the  CM  or  from  various  other 
subjective  measures  such  as  student  interviews  and  essays.  This  research  highlights  the 
properties  of  the  two  admissions  factors  and  the  estimated  impact  on  student 
performance.  The  results  show  student  performance  increased  as  CM  and  RAB 
increased,  revealing  the  importance  of  a  combined  quantitative  and  qualitative 
admissions  process  and  emphasizing  the  qualitative  input  as  the  value  added  to  the 
admissions  process  providing  the  increased  predictability  of  student  success. 
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I.  INTRODUCTION 


A.  BACKGROUND 

The  United  States  Naval  Aeademy  has  a  long  history  of  being  one  of  the  most 
highly  seleetive  universities  in  the  eountry.  They  reeeive  thousands  of  applieations  a  year 
and  ultimately  reduce  the  number  to  roughly  1,250  highly  competitive  candidates  who 
raise  their  right  hand  on  Induction  Day  to  join  the  Brigade  of  Midshipmen.  This 
impressive  process  of  admissions  is  geared  for  admitting  highly  qualified  students  who 
are  well  rounded  and  are  most  likely  to  achieve  success  at  the  United  States  Naval 
Academy.  But  what  sets  the  United  States  Naval  Academy  apart  from  other  highly 
selective  colleges  and  universities  is  that  the  United  States  Naval  Academy  must  also  find 
those  who  are  motivated  for  service  in  the  United  States  Navy  and  Marine  Corps. 

The  Naval  Academy  must  carefully  review  the  applicants  and  admit  only  those 
candidates  who  are  most  likely  to  succeed  as  students  at  the  Naval  Academy  and  who 
will  serve  honorably  and  dutifully  in  the  fleet  after  they  graduate.  With  this  larger  vision 
of  service  in  mind,  the  Admissions  Board  must  admit  the  most  qualified  candidates  and 
these  candidates  must  also  fit  the  institution  and  its  needs.  To  find  the  most  qualified 
candidates  it  starts  with  an  exhaustive  admissions  package  submitted  by  the  candidates. 
This  package  provides  information  about  the  candidate  on  all  aspects  of  their  educational 
experience  in  easily  quantifiable  measures  or  measures  that  are  easily  normalized  into 
quantifiable  data.  The  admissions  board  tallies  all  of  the  candidates’  qualifications  into 
what  is  referred  to  as  the  Candidate  Multiple.  This  multiple  is  based  on  a  weighted 
algorithm  where  all  the  applicant  data  that  can  be  quantified  is  plugged  in.  The  selected 
weights  and  factor  are  based  on  many  years  of  selecting  quality  candidates  to  the  Naval 
Academy  and  ultimately  producing  Navy  and  Marine  Corps  officers.  The  algorithm 
computes  the  Candidate  Multiple  and  this  is  the  first  tier  of  qualification  established  by 
the  Admissions  Board.  The  candidate  must  have  a  minimum  Candidate  Multiple  to  be 
considered  by  the  Admissions  Board  for  review. 

Once  it  has  been  determined  that  the  candidate  has  met  minimum  qualification 
and  is  fully  medically  qualified,  the  Admissions  Board  reviews  the  applicant’s  package  in 
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its  entirety.  Upon  review  of  the  package,  if  the  Admissions  Board  sees  something 
noteworthy  about  the  candidate  that  is  not  captured  by  the  quantitative  data,  it  may  award 
a  Recommendation  of  the  Admissions  Board  (RAB)  which  consists  of  raw  points  that  are 
added  to  the  Candidate  Multiple. 

The  Candidate  Multiple  and  the  RAB  points  are  summed  to  create  the  Whole 
Person  Multiple.  The  Whole  Person  Multiple  is  the  score  that  determines  whether  a 
candidate  is  considered  fully  qualified  and  ultimately  competitive  for  appointment  to  the 
Naval  Academy.  The  Whole  Person  Multiple  captures  observable  traits,  like  SAT  scores 
and  high  school  grade  point  average,  as  well  as  capturing  unobservable  traits,  like  a 
strong  teacher  recommendation,  an  outstanding  personal  essay  or  possibly  a  challenging 
experience  requiring  persistence  and  perseverance  above  and  beyond  that  of  the  average 
candidate.  This  method  provides  a  holistic  approach  to  find  the  candidates  to  fill  the 
ranks  of  the  Brigade  of  Midshipmen  that  will  go  on  to  serve  in  the  Navy  and  Marine 
Corps.  The  Admissions  Board  at  the  Naval  Academy  has  thousands  of  qualified 
applicants  each  year  who  go  through  the  admissions  process  resulting  in  a  historical 
graduation  rate  of  approximately  eighty  percent  which  ultimately  provides  candidates 
who  were  successful  Midshipmen  and  they  continue  to  serve  our  country  with  honor  in 
the  Armed  Forces,  the  government  and  in  our  local  communities. 

B,  PURPOSE 

The  purpose  of  this  research  is  to  examine  the  role  of  the  Recommendations  of  the 
Admissions  Board  (RAB)  scores  in  providing  value  added  to  the  admissions  process  in 
predicting  which  applicants  are  best  suited  for  the  Naval  Academy.  The  RAB  has  played 
a  vital  role  in  the  overall  admissions  process  and  is  the  final  opportunity  for  the 
Admissions  Board  to  award  points  to  an  applicant.  These  extra  points  are  something  the 
Admissions  Board,  after  a  comprehensive  review  of  the  individual  package,  awards  the 
candidate  for  mostly  unobservable  traits.  This  research  will  attempt  to  validate  the 
Admissions  Boards  statistical-based  scoring  model  and  the  value  added  of  the  RAB  score 
to  the  predictive  power  of  the  model.  The  study  will  use  indicators  of  success  of 
admitted  applicants,  such  as,  graduation  and  order  of  merit. 

The  following  research  questions  are  examined  by  this  thesis. 
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1.  How  does  the  Admissions  Board  utilize  the  RAB  in  the  admissions 
process? 

RAB’s  are  awarded  to  candidates  who  the  Admissions  Board  feels  have 
demonstrated  exceptional  potential  for  leadership  and  future  success  at  the  Naval 
Academy  through  various  subjective  measures  such  as  determination,  character,  and 
experience.  These  subjective  measures  are  characterized  as  unobservable  traits,  as  they 
are  not  captured  by  the  quantitative  measures  included  in  the  Candidate  Multiple.  The 
RAB  captures  those  unobservable  traits  and  adds  points  to  the  Candidate  Multiple  to 
create  the  Whole  Person  Multiple. 

2,  Using  a  multivariate  regression  modeling  approach,  can  we  model  the 
determinants  of  the  RAB  and  validate  the  value  it  adds  to  the  admissions  process? 

Using  the  available  data,  which  includes  the  graduation  outcome  of  each 
midshipman  and  other  selected  performance  measures,  we  will  develop  a  model  to 
predict  an  applicant’s  success  and  to  validate  the  RAB  as  a  predictor  of  that  success. 

C.  METHODOLOGY 

We  have  obtained  historical  data  from  the  Office  of  Institutional  Research  at  the 
Naval  Academy  representing  candidate  admissions  data  from  the  classes  of  1995  through 
2001.  Following  the  discussion  of  descriptive  statistics,  we  build  a  regression  model  of 
the  role  of  RAB  scores  as  a  predictor  of  success  at  the  Naval  Academy.  Using  various 
measures  of  success  we  attempt  to  validate  the  RAB  as  a  predictor  of  student  success. 

D,  ORGANIZATION  OF  STUDY 

Chapter  II  reviews  prior  studies  of  the  admissions  process  at  selective  college  and 
universities.  With  the  rise  in  applicants  trying  to  enter  colleges  and  universities, 
admissions  officers  have  been  speaking  out  and  trying  to  find  a  way  to  handle  all  of  the 
applications  and  to  keep  the  institutions’  needs  in  line  with  student  demands.  This 
balance  lends  itself  to  maintaining  humanistic  aspects  of  college  admissions  while 
finding  ways  to  streamline  the  application  evaluation  process.  Not  only  is  streamlining 
the  admissions  process  a  challenge  for  current  admissions  officers  but  finding  a  way  to 
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maintain  diversity  on  the  eampus  has  also  been  an  extremely  eomplex  nut  to  erack.  This 
seareh  for  a  balance,  guides  our  literature  review,  which  surveys  different  admissions 
theories,  processes  and  opinions. 

Chapter  III  presents  the  quantitative  data  obtained  from  the  Naval  Academy’s 
Office  of  Institutional  Research  for  cohorts  1995  through  2001.  This  chapter  discusses 
descriptive  statistics  of  the  data  sample.  From  the  data,  measurable  traits  emerge  that  can 
be  used  in  the  regression  analysis.  We  will  explain  the  performance  measures  we  have 
chosen  to  use  in  the  validation  process.  Based  on  the  statistical  scoring  model  of  the 
Naval  Academy  admissions  process,  we  then  conduct  a  regression  analysis  to  validate  the 
RAB  as  a  predictor  of  success  at  the  Naval  Academy.  Chapter  IV  analyzes  the  results  of 
the  regression  model  and  validates  the  RAB  as  a  predictor  of  success  at  the  Naval 
Academy. 

Chapter  V  concludes  the  study  and  evaluates  the  value  added  of  RAB  scores  in 
the  Naval  Academy’s  statistical-based  admissions  process.  This  chapter  will  also  provide 
recommendations  based  on  the  results  of  the  study  to  the  Admissions  Board  as  well  as  lay 
out  options  for  future  studies  in  this  area. 
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II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

The  continuing  efforts  of  admissions  officials  to  create  an  admissions  process  that 
balances  the  subjectivity  and  humanistic  factors  with  the  use  of  quantitative  data  in 
selecting  applicants  has  been  a  strategic  battle  for  years  within  highly  selective  U.S. 
colleges  and  universities.  During  the  last  decade  the  admissions  processes  for  highly 
selective  college  and  universities  have  been  feeling  the  pressure  from  the  public  because 
of  a  dramatic  increase  in  the  total  number  of  applicants,  particularly  from  the  “Baby 
Boom  Echo,  the  large  cohort  of  children  of  the  Baby  Boomers,”  (Long,  2003)  and 
numerous  other  admissions-related  issues  that  have  been  in  the  front  lines  of  the  press. 
Naturally,  officials  have  been  pressured  to  discuss  implications  and  impacts  of  recent 
events  and  trends  in  their  admissions  arena. 

The  Supreme  Court  ruling  in  June  of  2003,  upheld  race-based  admissions  in 
defense  of  affirmative  action  which  “carefully  circumscribed  authorization  to  continue  a 
practice  that  almost  all  consider  valuable.”  (Bok,  2003)  It  was  this  ruling  that  has 
admissions  officials  re-examining  their  own  admissions  policies  to  ensure  they  continue 
to  further  minority  enrollment.  In  seeking  minority  enrollment,  selective  colleges  and 
universities  continually  look  for  ways  to  identify  minority  students  who  have  qualities 
and  traits  that  are  predictors  of  success  at  their  institutions.  “It  is  under  these  conditions 
that  racial  preferences  in  higher  education  have  been  attacked  most  severely  as  the 
country  debates  how  to  distribute  access  to  competitive,  four  year  colleges.”  (Long, 
2003)  This  affects  the  screening  process  to  find  those  minority  students  and  it  is  evolving 
into  investigating  factors  beyond  the  standardized  test  scores  and  high  school  grade  point 
averages.  Because  of  a  growing  gap  in  minority  achievement  on  standardized  test  scores 
and  a  lower  average  high  school  grade  point  average  for  minorities,  the  student  essays 
and  interviews  are  making  strong  cases  as  well  as  teacher  and  guidance  counselor 
recommendations  in  helping  to  predict  minority  success. 

In  addition  to  affirmative  action  and  diversity  issues  facing  admission  officials, 
the  admissions  system  is  becoming  chaotic. 
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The  system  has  become  chaotic  because  it  is  overloaded.  Changes  in 
demographics,  technology,  and  society  have  saddled  the  most  selective 
colleges  with  more  applicants  than  they  know  how  to  handle.  More 
applicants  mean  an  even  higher  rate  of  rejections,  which  makes  a  college 
statistically  more  ‘selective.’  Perversely,  this  makes  it  all  the  more 
attractive  to  the  next  crop  of  applicants  and  the  cycle  goes  on.  (Fallows  et 
ah, 2003) 

With  the  increase  in  the  number  of  applicants  in  a  chaotic  system,  the  pressure  on 
the  admissions  process  increases.  These  highly  selective  colleges  and  universities  must 
choose  which  of  the  applicants  will  receive  one  of  the  limited  slots.  With  the  increased 
competition  and  pressure,  the  burden  lies  on  the  admissions  board  to  choose  wisely,  and 
to  offer  entry  to  those  students  who  they  feel  will  succeed  at  their  institution  as  well  as 
meet  the  needs  of  the  school,  while  simultaneously  meeting  the  needs  of  the  students. 

The  admissions  process  for  highly  selective  college  and  universities  has 
traditionally  been  a  topic  held  very  close  to  the  vest,  until  recently.  Admissions  officers 
have  become  more  open  and  willing  to  talk  about  their  experiences  as  well  as  weaknesses 
in  the  arena  of  the  admissions  system.  Academic  journal  articles,  forums  and  editorials 
are  exposing  many  of  the  issues  the  admissions  officials  face  and  what  solutions  are 
being  proposed  to  fix  them.  With  all  this  exposure,  college  admissions  officers  are 
feeling  the  increasing  pressure  from  the  growing  college  bound  high  school  senior 
population. 

Over  time,  the  range  of  qualifications  necessary  to  be  considered  for  admittance 
to  a  highly  selective  college  or  university  has  become  more  complex.  Admissions 
officers  now  use  both  quantitative  and  qualitative  indicators  to  screen  applicants.  The 
quantitative  measures  include  such  factors  as  standardized  test  scores  and  normalized 
high  school  grade  point  averages.  The  most  prolific  of  the  measures  are  the  SAT  and 
ACT,  both  of  which  are  standardized  tests  that  are  administered  to  high  school  juniors 
nationwide.  These  standardized  tests  have  grown  in  value  over  the  years  and  have 
become  somewhat  of  a  cornerstone  in  the  admissions  process.  They  also  have  enabled 
admissions  processes  to  use  quantitative  models  to  screen  applicants.  However,  highly 
selective  colleges  and  universities  are  starting  to  realize  that  admitting  well  rounded 
students  requires  more  than  admitting  those  scoring  in  the  top  percentile  on  the 
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standardized  tests  and  with  the  highest  grade  point  averages.  The  ehange  we  are  seeing 
today  is  the  movement  toward  an  inereased  use  of  a  mixed  approach  which  includes  both 
qualitative  and  quantitative  measures  to  capture  the  attributes  of  the  well  rounded  student. 


B,  QUANTITATIVE  MODELS  OF  A  QUALITATIVE  ADMISSIONS 

PROCESS 

Sadler  and  Hammerman  (1999)  discuss  how  quantitative  models  for  selecting 
Harvard  Graduate  School  applicants  can  save  time,  reduce  bias  and  strengthen  their 
graduate  program.  This  study  illustrates  how  quantitative  modeling  is  “used  to  predict  a 
reasonable  cut-off  for  selection  of  candidates  who  should  be  considered  in  the  next 
stage.”  (Sadler  and  Hammerman,  1999)  The  importance  of  this  research  to  us  is  the  use 
of  both  quantitative  modeling,  as  well  as  qualitative  subjective  measures  to  predict  which 
candidates  advance  to  the  next  stage. 

Harvard’s  graduate  admissions  process  is  different  from  most  undergraduate 
admissions  processes  because  of  the  composition  of  the  board,  which  includes  both 
faculty  and  students.  In  addition,  the  mindset  of  the  board  is  different  from  most 
undergraduate  programs  because  the  doctoral  student  is  seen  as  a  long  term  investment 
by  the  university.  The  admissions  process  at  the  Harvard  Graduate  School  of  Education 
may  not  be  an  exact  match  to  the  Naval  Academy’s  process  because  USNA  does  not 
have  students  on  the  board  of  admissions.  However,  both  processes  are  similar  in  that 
they  view  the  admissions  decision  as  a  long  term  investment.  For  example,  the  Naval 
Academy  must  select  young  men  and  women  to  ultimately  serve  in  careers  as  officers  in 
the  US  Navy  and  Marine  Corps. 

The  Saddler  and  Hammerman  (1999)  study  was  conducted  over  a  five-year  period 
“characterizing  a  three-stage  admissions  process  that  relies  heavily  on  judgments  of 
quality  based  in  complex  data.”  In  the  first  stage,  individual  members  read  the 
admissions  packages  and  rate  each  candidate.  In  the  second  stage  the  candidates  are 
discussed  by  the  committee  and  rated  through  group  consensus.  In  the  third  stage,  the 
candidates  are  compared  and  each  case  is  decided.  (Sadler  and  Hammerman,  1999) 
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This  three-step  proeess  uses  quantitative  modeling  but  relies  heavily  on  a 
subjective  review  of  each  candidate’s  admissions  package,  which  includes  prior  success 
history,  recommendations  and  test  scores.  The  subjective  review  looks  to  extract  a 
“candidate’s  potential  for  educational  leadership,  depth  of  educational  ideas,  match  with 
the  program’s  strengths  and  resources,  and  motivation  for  embarking  on  a  doctoral 
program.”  (Sadler  and  Hammerman,  1999)  This  subjective  review  extracts  traits  that  are 
not  captured  in  any  statistical  based  model.  This  identification  of  the  traits  that  match  the 
ideals  of  the  institution  is  the  very  same  thing  that  is  captured  by  the  RAB  at  USNA.  The 
Harvard  Graduate  School  of  Education  admissions  board  uses  the  quantitative  models  to 
narrow  the  field  of  candidates  and  then  reviews  the  packages  and  searches  for  desirable 
traits  that  predict  career  success.  The  Naval  Academy  Admissions  Board  brings  the  same 
value  added  to  their  statistical  based  scoring  model.  Once  the  quantitative  model  has 
narrowed  the  candidate  field,  the  Board  extracts  the  desirable  traits  and  quantifies  those 
traits  by  awarding  additional  points  to  the  Candidate  Multiple. 

The  overall  concepts  and  findings  of  the  Harvard  study  are  worthwhile  and 
pertinent  to  the  study  of  the  RAB  at  the  Naval  Academy  admissions  process.  Saddler  and 
Hammerman  show  that  even  though  quantitative  models  are  used,  the  subjective 
identification  of  the  institutionally  desired  traits  demonstrated  by  the  candidate  keeps  the 
focus  on  the  human  aspect  of  the  admissions  process.  The  study  illustrates  the  value  of 
combining  the  strengths  of  quantitative  modeling  with  a  largely  qualitative  admissions 
process. 

The  Harvard  admissions  process  is  “viewed  as  an  opportunity  to  match  the 
resources  and  needs  of  a  school  with  an  applicant’s  interests  and  talents,  impressions  and 
intuitions  must  substitute  for  the  comfort  of  numerical  scores.”  (Sadler  and  Hammerman, 
1999)  Similarly,  the  Naval  Academy  admissions  process  is  viewed  as  an  opportunity  to 
match  the  resources  and  needs  of  the  institution  with  a  candidate’s  interest  and  talents, 
where  impressions  and  intuitions  are  the  value  added  to  the  admissions  process  and 
quantified  in  the  form  of  a  RAB,  which  is  added  to  the  statistical  based  scoring  model. 
The  RAB  will  never  substitute  for  the  overall  numerical  scoring  model  but  it  certainly  is 
that  little  bit  extra  that  could  push  the  board  to  accept  candidates  with  lower  scores. 
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C.  SEEKING  DIVERSITY:  HOW  DO  WE  STRENGTHEN  DIVERSITY  AT 
HIGHLY  COMPETITIVE  CAMPUSES? 

In  June  2003,  there  were  two  Supreme  Court  ease  deeisions  that  involved 
diversity  issues  at  the  University  of  Miehigan.  The  ruling  upheld  one  admission  poliey 
and  turned  down  another.  In  the  case  the  Supreme  Court  upheld,  it  ruled  that  the  law 
school’s  admissions  policy  that  “had  compelling  interest  in  enrolling  a  racially  diverse 
student  body  because  of  educational  benefits  that  diversity  provides.  The  majority  said 
that  the  law  school’s  race-conscious  admissions  policy  was  an  acceptable  means  of 
achieving  that  diversity  because  it  considered  race  as  just  one  of  several  factors  in 
evaluating  each  individual.”  (Schmidt,  2003)  In  the  case  that  was  not  upheld,  the 
Supreme  Court  ruled  that  the  admissions  policy  relied  primarily  on  a  statistical-based 
scoring  system  that  was  “too  formulaic  and  mechanistic,  and  treated  whole  groups  of 
applicants  differently  based  solely  on  their  race.  Because  the  policy  was  not  ‘narrowly 
tailored’  to  achieving  educational  diversity  it  violated  the  Constitution’s  equal-protection 
clause  and  Title  VI  of  the  Civil  Rights  Act  of  1964,  which  prohibits  racial  discrimination 
by  any  institution,  public  or  private,  that  receives  federal  funds.”  (Schmidt,  2003)  The 
theme  the  Supreme  Court  was  ruling  on  was  that  race  needed  to  be  “evaluated  flexibly” 
and  not  used  by  treating  individuals  as  members  of  a  particular  racial  grouping. 
Beginning  with  the  freshman  class  entering  in  the  fall  of  2004,  university  officials  say 
they  will  “instead  consider  race  in  a  more  ‘holistic’  way.”  (Cavanagh,  2003)  The  new 
University  of  Michigan’s  undergraduate  admissions  policy  is  serving  as  a  “harbinger  of 
admissions  overhauls... where  administrators  are  seeking  to  craft  legally  defensible 
affirmative  action  plans  of  there  own.”  (Cavanagh,  2003) 

College  admissions  officials  read  the  Supreme  Court  rulings  as  allowing  race- 
based  admissions  in  higher  education  as  long  as  they  institute  a  more  comprehensive 
evaluation  system,  which  measures  the  abilities  of  each  applicant  on  an  individual  basis. 
The  challenge  for  the  college  and  universities  comes  in  the  manner  of  how  do  they 
identity  the  qualified  minorities  that  are  seeking  admission  to  their  respective  university. 

Universities  can  also  seek  new  ways  to  identify  minority  high-school 
students  with  personal  qualities  that  would  allow  them  to  overcome 
modest  grades  and  test  scores  and  succeed  in  college.  The  problem  is  not, 
as  some  liberal  critics  continue  to  assert,  that  minority  scores  are 
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artificially  low  because  of  cultural  bias  in  the  tests.  In  faet,  the  reverse  in 
true:  Standardized  tests  consistently  over-predict  the  academic 
performance  of  minority  students  in  college  and  professional  school. 

Even  so,  tests  and  high-sehool  reeords  are  far  from  infallible;  many 
minority  students  ean  do  mueh  better  than  their  prior  reeords  would 
prediet.  If  admissions  committees  eould  identify  more  of  those  young 
people,  larger  numbers  of  poor  and  minority  students  could  gain  access  to 
seleetive  eolleges.  (Bok,  2003) 

D.  THE  IMPLICATIONS  OF  TRADITIONAL  MEASURES:  SAT,  GPA, 
STUDENT  ESSAYS,  INTERVIEWS,  COUNSELOR  STATEMENTS. 

The  standardized  test  has  been  under  scrutinized  by  statisticians  and  researchers 

for  years  as  they  have  been  trying  to  find  the  appropriate  fit  into  admissions  proeess. 

Keller,  Crouse,  and  Trushiem  (1994)  published  a  study  that  explored  the  effeet  of  the 

SAT  on  admissions  and  the  results  suggest  that  the  SAT  has  more  of  an  effeet  on  the 

composition  of  a  freshman  class  than  it  does  the  aeademie  outeomes  of  the  class.  “The 

faet  that  the  SAT  has  compositional  effects  means  that  most  colleges  probably  ignore  the 

SAT  seores  of  some  applicants  to  balance  the  composition  of  their  freshman  class.  They 

must  do  this  when  they  eonsider  finaneial  need  and  affirmative  action  in  their 

admissions.”  (Keller  et  ah,  1994)  “The  eompositional  effeets  of  the  SAT  eould  inerease 

the  number  of  engineering  majors,  but  deerease  the  number  of  edueation,  human 

resourees,  nursing,  and  physieal  edueation  majors.  It  eould  also  decrease  female  and 

blaek  admissions.”  (Keller  et  ah,  1994) 

Standardized  tests  scores  alone  should  not  be  the  sole  faetor  in  which  an 
admissions  decision  is  made.  Carole  Veir  (1990)  presented  a  paper  at  the  Annual 
Meeting  of  the  University  Couneil  of  Educational  Administrators  discussing  how  to 
identify  potential  leaders  through  pre-admittance  assessment.  In  her  paper,  she  explains 
how  the  Eeadership  Assessment  Center  proeess  utilized  at  the  University  of  Texas  at 
Austin  selects  students  for  the  Educational  Administration  Leadership  Program.  The 
assessment  eenter  was  finding  that  they  were  admitting  students  who  did  not  possess  the 
ability,  personality  or  potential  to  be  leaders  for  the  schools.  After  studying  and 
reviewing  their  admissions  proeess  they  were  finding  that  they  were  admitting  students 
based  on  their  standardized  test  seores.  There  was  no  humanistie  or  subjeetivity  in  the 
admission  proeess.  Erom  that  point  on  the  Leadership  Assessment  Center  learned  that  “a 
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good  GRE  score  alone  does  not  denote  potential  for  leadership,  but  rather  potential  to 
sueeeed  in  graduate  programs.”  (Veir,  1990)  This  realization  turned  them  baek  to  their 
admissions  proeess  and  they  “determined  that  program  admission  should  be  determined 
by  multiple  sources  of  evidenee  that  include  much  more  information  than  these  standard 
measures,  whieh  provide  little  information  about  the  most  essential  aspeets  of  the  ability 
to  be  a  sehool  leader  and  survive  a  rigorous  graduate  program  with  a  strong  foeus  on 
eommunieation  and  interpersonal  skills,  leadership  and  deeision-making  proeedures.” 
(Veir,  1990) 

Even  today  as  the  admissions  proeess  moves  towards  goals  of  an  inereasingly 
diverse  student  body,  the  effeet  of  standardized  test  seores  is  still  an  issue  of  debate.  But 
as  the  debate  eontinues,  admission  offieials  are  seeking  the  appropriate  balanee  of 
quantitative  data  through  standardized  tests  and  elusive  data  on  more  subjeetive 
measures.  “Admissions  eommittees,  particularly  at  these  types  of  institutions  [highly 
selective],  take  into  aeeount  a  wide  variety  of  eriteria,  and  some  of  these  factors  are  likely 
to  be  subjective  measures  not  easily  eaptured  in  analysis.  Eor  example,  many  schools 
require  student  essays  and  recommendations  from  teaehers.  Moreover,  extraeurrieular 
aetivities  and  leadership  experienees  are  also  important  influenees  in  applieation 
deeisions.”  (Eong,  2003) 

Some  offieials  are  more  holistie  in  nature  looking  for  that  distinguishing  attribute 
that  the  applieant  will  eontribute  to  the  institution.  “As  an  admissions  officer,  I  looked 
for  clues  to  charaeter.  What  has  the  student  done  to  overeome  obstacles?  In  what  ways 
has  the  student  distinguished  herself?  These  items  ean  be  revealed  in  ways  that  eould  be 
mueh  more  student-e entered  and  effieient.”  (Sjogren,  2004) 

In  a  paper  to  the  Assoeiation  for  Institutional  Research  in  1989,  Kanarek 
discusses  how  Willingham  (1985)  found  that  “’produetive  flow  through’  in  high  school 
‘purposeful,  eontinuous  eommitment  to  eertain  types  of  aetivities  versus  sporadie  efforts 
in  diverse  areas’  was  the  best  predietor  of  overall  sueeess.  He  notes,  however,  that 
‘extraeurrieular  productivity’  is  not  a  substitute  for  aeademie  qualifieation.”  In  eontrast, 
she  also  notes  that  Trushiem  and  Middaugh  (1987)  “found  that  personal  qualities  were 
not  related  to  the  prediction  of  freshman  grades.”  He  also  suggests  that  the  eollection  of 
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personal  data  is  an  inefficient  and  an  unjustifiable  contribution  to  the  prediction  of 
freshman  grades.  So  the  bottom  line  is  determined  by  the  characteristics  the  Admission 
Board  is  searching  for  in  their  student  body.  Are  they  screening  for  leaders  and  decision 
makers,  or  are  they  screening  to  predict  freshman  grade  point  averages?  The 
concentration  and  focus  of  the  Admissions  Board  is  critical  to  what  data  is  collected  and 
how  it  is  weighed  in  the  process. 

In  addition  to  the  data  collection,  the  interview  process  and  “developing  regional 
alumni  interviewing  committees  is  another  way  institutions  have  managed  to  meet  more 
students  face-to-face.”  (Greene  et  al.,  2003)  “As  long  as  they  are  conducted  sensitively, 
interviews  can  be  an  important  part  of  a  holistic  admissions  process.”  (Greene  et  al., 
2003) 
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III.  DATA  AND  METHODOLOGY 


A,  DATA  SOURCE 

This  study  draws  on  data  obtained  from  the  Offiee  of  Institutional  Researeh  at  the 
Naval  Aeademy.  The  dataset  eontains  eandidate  applieation  information,  sueh  as 
demographics  and  high  school  performance  as  well  as  midshipmen  performance 
information.  Using  cohorts  entering  for  graduation  years  between  1995  and  2001  yields  a 
total  of  8,299  individual  records,  6,495  of  which  are  graduate  records.  From  the  6495 
individual  graduate  records,  65  individual  records  were  removed  leaving  6,430  individual 
cases  for  estimating  the  order  of  merit,  academic  QPR  and  military  QPR  outcome 
measures.  For  estimating  the  striper  selection  outcome,  90  individual  graduate  records 
were  removed  from  the  6,495  individual  graduate  records  for  missing  data.  Figure  1 
represents  the  total  number  of  cases  (accepted  applicants)  by  graduating  class  year  in  the 
data  sample. 


Graduating  Class  Year 

Figure  1,  Number  of  Accepted  Applicants  in  Each  Graduation  Year 


13 


The  Office  of  Institutional  Research  does  not  maintain  information  on  candidates 
who  do  not  attend  the  Naval  Academy.  The  data  set  did  not  contain  information  for 
candidates  who  were  offered  an  appointment  and  opted  not  to  attend  the  Naval  Academy 
or  for  candidates  who  were  not  offered  an  appointment.  This  is  important  to  keep  in 
mind  as  all  results  of  the  study  are  based  on  accepted  applicants  rather  than  the  full 
population  of  total  applicants. 

B,  METHODOLOGY 

To  examine  the  relationship  between  qualitative  judgments  of  the  Admissions 
Board  (RAB)  and  accepted  applicant  performance  at  USNA,  this  study  follows  several 
steps.  The  explanatory  and  dependent  variables  are  identified  and  defined  based  in 
information  in  the  dataset  received  from  the  USNA  Office  of  Institutional  Research.  The 
independent  variables  and  the  development  of  the  multivariate  regression  models  were 
based  on  the  types  of  performance  measures  chosen  for  the  study. 

This  section  on  methodology  is  divided  into  two  parts:  (1)  method  of  analysis;  (2) 
outcome  performance  variables.  Several  factors  are  considered  in  defining  the 
independent  and  dependent  variables  used  in  this  study.  These  include:  how  to  categorize 
the  variables  (dichotomous,  categorical,  or  continuous);  correlations  between  explanatory 
variables;  and  the  possibility  of  using  interaction  variables. 

Linear  and  logistic  multivariate  regressions  are  used  to  analyze  relationships 
between  the  selected  performance  (outcome)  measures  and  the  subjective  judgments  of 
the  Admissions  Board  (RAB).  For  this  phase,  several  dependent  variables  are  selected 
as  outcome  performance  measures.  These  include:  graduation  (Grad);  order  of  merit 
(Pctloom);  cumulative  academic  quality  point  ratio  (Cumaqpr);  cumulative  military 
quality  point  ratio  (Cummqpr);  and  striper  selection  (Stripers). 

1.  Method  of  Analyses 

We  will  address  the  modeling  analyses  primarily  focusing  on  the  groups  which 
we  could  consider  a  trade  off  in  overall  performance  for  diversity.  The  regression  models 
will  use  two  types  of  multivariate  estimation  techniques  to  quantify  the  relationships 
between  the  RAB  and  the  selected  performance  measures.  Standard  Ordinary  Least 

Squares  (OLS)  regression  analysis  is  used  for  models  where  the  dependent  variables  are 
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continuous,  such  as  PctlOOM,  Cumaqpr,  and  Cummqpr.  Logistic  (Logit)  regression 
analyses  are  used  for  binary  dependent  variables,  in  our  ease  Grad  and  Stripers. 
Deseriptions  of  the  OLS  regression  and  the  Logit  regression  models  are  included  in  the 
following  two  subsections. 

a.  General 

Linear  multivariate  regression  analysis  is  used  for  a  number  of  reasons. 
First,  multiple  regression  teehniques  can  be  applied  to  a  data  set  where  the  independent 
variables  are  somewhat  correlated  with  one  another  and  with  the  dependent  variables  to 
varying  degrees.  Second,  multiple  regression  uses  several  independent  variables  to 
predict  the  dependent  variable.  Third,  the  result  of  the  estimated  regression  model  is  an 
equation  that  represents  the  best  prediction  of  a  dependent  variable  from  several 
(continuous  or  dichotomous)  independent  variables.  The  regression  equation  takes  the 
following  form: 


Y'  =  A  +P1X1  +P2X2  +  ...+  PkXk+e  (1) 


Where  Y'  is  the  predicted  of  the  dependent  variable,  A  is  the  Y  intereept, 
the  Ys  represent  the  various  independent  variables,  the  Bs  are  the  coefficients  of  eaeh  of 
the  independent  variables  to  be  estimated,  and  e  is  the  stoehastic  error  term.  The 
estimated  intercept  and  coefficients  are  used  to  predict  the  values  of  the  dependent 
variables  for  all  observations  in  the  sample.  A  different  Y'  value  is  predicted  for  each 
observation  as  a  result  of  inserting  the  subject’s  own  Y  values  into  the  equation. 

The  goal  of  the  regression  is  to  estimate  the  p  values,  ealled  eoefficients, 
for  the  independent  variables  that  minimize  the  difference  between  the  Y  values  predicted 
from  the  equation  and  the  Y  values  obtained  by  measurement.  That  is,  ordinary  Least 
Squares  teehniques  minimize  deviations  between  predieted  and  obtained  Y 

values  and  optimize  the  correlation  between  the  predieted  and  obtained  Y  values  for  the 
data  set.  (See  Tabaehnick,  Barbara  G.  Using  Multivariate  Statistics,  4**^  ed.  Ch5,  pill- 
112) 


15 


In  cases  where  a  clear  (binary)  division  can  be  made  between  sueeessful 
performance  and  lower  performance,  such  as  graduation  (Grad)  or  selection  to  a 
leadership  position  that  rates  wearing  stripes  (Stripers),  linear  probability,  probit,  and 
logit  models  ean  all  be  used.  While  all  these  teehniques  are  appropriate  when  estimating 
a  relationship  between  a  set  of  explanatory  variables  and  a  diehotomous  (binary) 
dependent  variable,  this  study  uses  the  logit  regression  model.  The  logit  model  has 
neither  assumptions  about  the  distribution  of  the  predietor  variables  nor  do  the  predictors 
have  to  be  normally  distributed,  linearly  related,  or  of  equal  varianee  within  eaeh  group. 
The  predictors  can  be  a  mix  of  eontinuous,  diserete  and  diehotomous  variables.  Also,  the 
estimated  logit  model  cannot  produce  negative  predicted  probabilities. 

Because  the  logit  model  is  non-linear,  the  equations  used  to  deseribe  the 
outcomes  are  slightly  more  eomplex  than  those  for  the  OLS  multiple  regressions.  The 
outcome  variable,  Y  ,  is  the  probability  of  having  one  outeome  or  another  based  as  a 
nonlinear  function  of  the  best  linear  combination  of  predictors:  with  two  outcomes: 

F/=  e  /1+e 

where  Yis  the  estimated  probability  that  the  /th  ease  (1=1,  ...,«)  is  in  one 
of  the  categories  and  u  is  the  usual  linear  regression  equation: 


u  -  A  +  PiXi  +  P2X2  +  . . .+  PkXk 


with  constant  A,  coefficients  Pj,  and  predictors  Xj  for  k  predictors  (/  = 
1,2,. .  .,A:).This  linear  regression  equation  creates  the  logit  or  log  of  the  odds: 


Ln(f/l-f)=A+Y^PjXij 


The  linear  regression  equation  is  the  natural  log  (In)  of  the  probability  of 
being  in  one  group  divided  by  the  probability  of  being  in  the  other  group.  The  goal  is  to 
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find  the  best  linear  eombination  of  predictors  to  maximize  the  likelihood  of  obtaining 
observed  outcome  frequencies.  The  most  complex  and  best  fitting  model  includes  the 
constant,  all  predictors,  and  perhaps  interactions  among  predictors.  (See  Tabachnick, 
Barbara  G.  Using  Multivariate  Statistics,  4*  ed.  Ch5,  p5 17-5 19) 

b.  Modeling  Specifications 

(1)  Multiple  Regression  Analysis.  We  have  constructed  a  series 
of  regression  models  by  adding  explanatory  variables  to  each  additional 
regression  to  understand  and  illustrate  the  relationship  of  the  RAB  to  the  given 
performance  measure.  The  baseline  regression  model  includes  independent 
variables  for  gender  (Female),  minority  groups  (Aframer,  Hispanic,  and 
Othrace),  athletic  recruit  status  (Recblchp),  and  nomination  source  (Nomprvp, 
Nomqalt,  Nomenrtc,  and  Nomsupe). 

The  second  step  in  the  modeling  adds  candidate  multiple 
(Cmthous)  to  the  regression  as  an  independent  variable.  This  is  an  important  step  in  the 
analysis  because  we  now  account  for  the  candidate  multiple  in  the  regression  and  its 
relationship  to  the  performance  measure. 

The  third  step  adds  the  variable  representing  the  RAB  (RAB500) 
to  the  regression  model.  We  begin  to  see  the  impact  the  Admission  Board  has  once  they 
have  been  afforded  the  opportunity  to  review  each  candidate’s  package  and  award  RAB’s 
to  the  candidate. 

The  fourth  step  adds  a  variable  to  the  regression  model  that 
interacts  the  RAB  score  and  the  candidate  multiple  (RABCM).  This  is  to  test  for  any 
interaction  effect  between  the  RAB  and  the  candidate  multiple. 

The  fifth  and  final  step  of  our  regression  model  checks  for  non¬ 
linearity  by  adding  the  squared  candidate  multiple  variable  (CMThSqrd)  to  the  model. 
Also,  the  interaction  variables  between  RAB  score  and  the  four  distinct  quartiles  of  the 
candidate  multiple  (RABL058,  RAB5861,  RAB6165,  RAB65HI)  are  added.  In  this 
regression,  we  have  removed  the  RAB  *  Candidate  Multiple  (RABCM)  interaction 
variable.  The  steps  of  the  regression  model  and  the  sequence  in  which  the  independent 
variables  are  entered  into  the  model  is  summarized  in  Table  1  below. 
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Table  1.  Multiple  Regression  Models 


Step  1  Step  2  Step  3  Step  4  Step  5 


Gender 

X 

X 

X 

X 

X 

Ethnicity 

African  American  (Aframer) 

X 

X 

X 

X 

X 

Hispanic  (Hispanic) 

X 

X 

X 

X 

X 

Other  Minority  (Othrace) 

X 

X 

X 

X 

X 

Athletic  Status 

Blue  Chip  (Recblchp) 

X 

X 

X 

X 

X 

Nomination  Source 

President  &  VP  (Nomprvp) 

X 

X 

X 

X 

X 

Qualified  Alternate  (Nomqalt) 

X 

X 

X 

X 

X 

Enlisted  &  NROTC  (Nomenrtc) 

X 

X 

X 

X 

X 

Superintendent  USNA  (Nomsupe) 

X 

X 

X 

X 

X 

CM&RAB 

CM/ 1,000  (Cmthous) 

X 

X 

X 

X 

CM"(CMTHSqrd) 

X 

X 

X 

X 

RAB/500  (RAB500) 

X 

X 

Interactions 

CM/1000  *  RAB/500  (RABCM) 

X 

Interactions  Non-linear 

(RABL058) 

X 

(RAB5861) 

X 

(RAB6165) 

X 

(RAB65HI) 

X 

Note:  ‘x’  indicates  variable  indue 

ed  in  model. 

(2)  Logit  Regression  Analysis.  The  diehotomous  dependent 
variables  graduate  (Grad)  and  striper  seleetion  (Stripers)  are  estimated  using  a 
logit  model.  We  follow  the  same  five-step  proeess  in  our  logit  model  as  we  do  for 
our  OLS  models  discussed  above.  The  input  of  the  independent  variables  and 
interaction  variables  in  this  same  process  once  again  allows  us  to  measure  the 
impact  of  each  independent  variable.  The  independent  variables  used  in  our  logit 
models  for  our  two  dichotomous  dependent  variables  are  the  same  as  summarized 
in  Table  1. 

2,  Outcome  Performance  Measures 

The  outcome  performance  measures  are  chosen  to  parallel  research  found  in  the 
literature  regarding  admissions  and  performance  of  the  admissions  process.  Several 
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dependent  variables  are  selected  as  outcome  performance  measures,  including: 
graduation  (Grad),  percentile  order  of  merit  (Pctloom),  cumulative  academic  quality 
point  ratio  (Cumaqpr),  cumulative  military  quality  point  ratio  (Cummqpr),  and  striper 
selection  (Stripers)  and  are  discussed  in  detail  in  the  following  sections  below.  Table  2 
contains  descriptive  statistics  for  the  outcome  performance  measures. 


Table  2.  Outcome  Performance  Measure  Descriptive  Statistics 


OUTCOME  PERFORMANCE 
MEASURES 

N  of  Cases 

Mean 

Std  Dev. 

MIN 

MAX 

Graduate  (Grad) 

8299 

0.78 

0.4120 

0 

1 

Percentile  of  OOM  (Pctloom) 

6430 

50.05 

0.3600 

1 

100 

Cumulative  AQPR  (Cumaqpr) 

6430 

2.93 

0.0059 

2 

4 

Cumulative  MQPR  (Cummqpr) 

6430 

3.18 

■ilitilebM 

2.1 

3.99 

Selected  for  Striper  Billet  (Stripers) 

6305 

0.19 

■lIlIlMlM 

0 

1 

*From  the  6495  graduate  cases,  65  cases  were  removed  for  missing  data. 
**  From  the  6495  graduate  cases,  90  cases  were  removed  for  missing  data 


a.  Graduation  (Grad) 

Grad  is  a  dichotomous  variable  with  1  representing  whether  or  not  a 
candidate  graduated  from  the  Naval  Academy  and  0  representing  a  non-graduate.  For 
this  variable,  we  included  delayed  graduates  as  well.  The  over-arching  criterion  is 
whether  or  not  the  candidate  completed  the  course  of  study  and  was  awarded  a  diploma 
from  the  Naval  Academy.  Figure  2  charts  the  mean  graduation  rate  by  each  graduating 
class  year  in  the  study. 
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Figure  2,  Mean  Graduation  Rate  by  Graduation  Year 
b.  Percentile  Order  of  Merit  (Pctloom) 

This  variable  represents  the  pereentile  ranking  of  eaeh  eandidate  in  the 
graduating  class.  The  raw  Order  of  Merit  is  the  sum  of  cumulative  performance 
measures  calculated  to  determine  class  ranking  upon  graduation.  The  measure  contains 
universally  weighted  information  to  include  academic  grades,  conduct  grades,  military 
performance  grades,  and  physical  readiness  tests.  The  variable  used  in  this  study, 
however,  is  derived  from  stacking  and  ranking  the  raw  Order  of  Merit  (OOM)  into  a  new 
variable  (ROOM)  which  represents  the  stacked  rankings  of  only  graduating  midshipmen. 
The  variable  (ROOM)  is  then  divided  by  the  number  of  members  in  each  graduating 
class.  This  stacked  ranking  is  done  because  members  of  the  class  that  did  not  graduate 
may  have  held  an  order  of  merit  position  thus  affecting  the  graduating  member’s  final 
order  of  merit  performance  measure.  Our  goal  is  to  see  how  successfully  graduating 
members  performed.  When  interpreting  this  performance  measure,  the  estimated 
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coefficient  represents  an  impact  on  percentile  order  of  merit,  not  individual  spots  a 
candidate  will  move  up  or  down.  Note  the  Mean  of  the  variable  (Pctloom)  should  equal 
50  by  definition. 

c.  Cumulative  Academic  Quality  Point  Ratio  (Cumaqpr) 

This  is  a  continuous  variable  representing  the  eumulative  aeademic  grade 
point  average  the  candidate  achieved  while  at  the  Naval  Aeademy.  This  grade  point 
average  measure  includes  only  academic  courses  of  study.  Figure  3  charts  the  mean 
cumulative  academic  QPR  by  each  graduating  class  year.  The  evident  rising  trend  in 
academic  performance  indicates  that  there  could  be  increased  academic  grade  inflation 
for  successful  cohorts  in  our  study.  Though  we  should  not  dismiss  that  this  inflation 
exists,  we  will  not  attempt  to  prove  or  disprove  the  existence  of  the  inflation.  To  account 
for  the  inflation  we  include  class  year  dummy  variables  in  the  regression  models  as 
independent  variables  when  analyzing  cumulative  academic  QPR  (Cumaqpr). 


Figure  3,  Mean  CUMAQPR  by  Graduation  Year 
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d.  Cumulative  Military  Quality  Point  Ratio  (Cummqpr) 

This  is  a  continuous  variable  representing  the  eumulative  military  grade 
point  average  the  eandidate  aehieved  while  at  the  Naval  Aeademy.  This  grade  point 
average  ineludes  military  eourses  of  study  as  well  as  eonduet  and  military  performanee 
grades  throughout  the  four  years  at  USNA.  It  is  evident  in  Figure  4  that  there  is  a 
downward  trend  in  the  eumulative  military  QPR  for  the  cohorts  in  this  study.  This  very 
well  eould  stem  from  a  deliberate  lowering  of  military  grade  inflation  during  the 
graduating  elass  year  eohorts  of  our  study.  We  will  not  attempt  to  prove  that  this 
deflation  exists,  but  it  is  pointed  out  as  it  affeets  our  performanee  measures  when  we 
eompare  the  entire  data  sample.  For  this  reason  we  have  ineluded  the  elass  year  dummy 
variables  as  independent  variables  in  the  models  analyzing  eumulative  military  QPR 
(Cummqpr). 


Figure  4,  Mean  CUMMQPR  by  Graduation  Year 
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e.  Striper  Selection  (Stripers. 

This  is  a  dichotomous  variable  with  1  representing  whether  the  eandidate 
was  seleeted  for  a  leadership  position  within  the  Brigade  of  Midshipmen  that  was 
awarded  three  stripes  or  more.  Three  stripes  and  above  represents  unit  eommanders 
(eompany,  battalion,  regimental,  brigade),  eommand  staff  (battalion,  regimental,  and 
brigade),  as  well  as  varsity  sports  team  eaptains.  Beeause  stripes  are  awarded  eaeh 
semester  to  the  midshipmen,  the  (Stripers)  dependent  variable  aeeounts  for  the 
eandidate’s  highest  rank,  three  stripes  or  above,  during  the  two  semesters  students  are 
eligible  for  stripes.  This  ehoiee  to  use  three  stripes  and  above  was  made  beeause  in  order 
to  earn  three  stripes  or  above  the  midshipmen  are  subjeetively  seleeted  by  the  aetive  duty 
leadership  at  the  Naval  Aeademy. 

Figure  5  eharts  the  striper  seleetion  rate  by  graduation  year.  Note  the 
deerease  in  mean  striper  seleetion  rate  from  graduating  elass  year  1995  to  1997.  The 
number  of  “striper”  billets  in  the  Brigade  of  Midshipmen  is  a  fixed  number  eaeh  year. 
The  seleetion  rate  ean  vary  due  to  differing  elass  sizes  but  the  deerease  in  the  ehart  is 
mostly  aeeounted  for  by  a  signifieant  deerease  in  the  number  of  available  striper  billets 
from  grad  year  1995  to  1997  as  there  was  a  ehange  in  the  organizational  strueture  of  the 
Brigade  of  Midshipmen  from  36  eompanies  to  30  eompanies.  This  affeeted  the  number 
of  available  striper  billets  to  fill  as  there  was  a  loss  of  six  eompany  eommanders 
(equating  to  twelve  lost  leadership  striper  positions)  and  the  loss  of  a  Battalion  staff 
(equating  to  approximately  20  to  25  lost  leadership  striper  billets).  Onee  this  transition 
was  eomplete,  the  number  of  available  striper  billets  to  award  remains  eonstant  between 
1998  and  2001.  We  note  this  beeause  it  eertainly  affeets  our  striper  seleetion  rate  over 
the  entire  data  sample.  Onee  again  beeause  of  the  impaet  of  elass  size,  we  will  inelude 
the  elass  year  dummy  variables  in  the  model  to  aeeount  for  this  ehanging  rate  of  striper 
seleetion. 
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Figure  5,  Mean  Striper  Selection  Rate  by  Graduation  Year 

3,  Independent  Variables 
a.  Diversity  Groups 

This  section  examines  the  various  demographic  groups  that  may  be  given 
special  consideration  by  the  Admissions  Board.  An  analysis  of  these  groups  will  help  to 
explain  the  function  of  the  RAB  and  show  that  the  value  added  of  the  RAB  to  the 
admissions  process  stems  from  its  use  as  a  diversity  tool  and  as  predictor  of  success. 

This  section  addresses  why  these  demographic  groups  are  considered 
special  by  the  Academy  and  analyzes  any  performance  gaps  between  these  diversity 
groups  and  the  rest  of  the  sample  in  the  study.  The  literature  review  has  identified  certain 
groups  that  have  the  greatest  potential  for  being  treated  differently  by  admissions  boards 
across  the  country,  including  females,  racial  and  ethnic  minorities,  as  well  as  recruited 
athletes.  In  addition,  groups  that  are  specific  to  the  Naval  Academy  admissions  process 
that  have  potential  to  be  treated  differently  are  those  that  receive  a  special  nomination 
from  sources  that  include  enlisted  Navy  and  NROTC  nominations,  military  1  egacy 


24 


nominations  from  the  President  and  Viee  President  of  the  United  States,  Superintendent 
of  the  Naval  Academy  nominations,  and  the  qualified  alternate  category  nominated  from 
the  Admissions  Board. 

To  completely  grasp  the  scope  and  enormity  that  these  special  groups 
introduce  to  the  Admissions  Board  when  reviewing  applications,  we  have  complied  Table 
3  that  sums  the  maximum  possible  number  of  accepted  applicants  that  fall  into  a  special 
consideration  category.  This  table  does  not  double  count  any  case  that  could  fall  into 
multiple  categories.  For  example,  out  of  the  8,299  accepted  applications  for  this  period, 
1,273  were  female  applicants  leaving  only  male  applicants  for  counting  1,315  ethnic 
minority  cases.  Once  females  and  ethnic  minority  applicants  have  been  accounted  for, 
1 ,003  are  blue  chip  athletes.  So  if  the  applicant  is  not  female,  is  not  an  ethnic  minority  or 
a  blue  chip  athlete,  they  still  could  be  considered  in  one  of  the  nomination  categories. 
The  following  table  sums  the  number  of  cases  that  could  potentially  be  considered  a 
special  category  by  the  Admission  Board  and  surprisingly  67.5%  of  all  accepted 
applicants  were  in  at  least  one  special  consideration  group  when  their  admissions  package 
was  reviewed  by  the  Admission  Board. 

We  must  understand  that  not  only  are  the  majority  (67.5%)  of  all  accepted 
applicants  in  this  study  in  a  group  that  could  receive  special  consideration,  some  of  these 
special  consideration  groups  have  a  gap  in  observed  performance  based  on  high  school 
performance,  standardized  test  scores  and  the  other  performance  measures  that  are 
captured  in  the  candidate  multiple.  In  analyzing  these  gaps,  the  candidate  multiple  sets 
the  stage  of  our  methodology  as  it  is  the  initial  performance  measure  calculated  for  every 
candidate. 

We  use  t-tests  and  anova-tests  as  appropriate,  to  analyze  differences  in 
performance  for  the  demographic  groups.  We  run  these  tests  on  the  groups  using  the 
candidate  multiple  as  the  measure.  Identifying  differences  in  the  performance  of  each 
group  will  help  us  understand  the  dynamics  of  the  RAB  and  how  it  is  used  as  a  universal 
diversity  tool  and  some  cases  to  overcome  observed  performance  gaps. 
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Table  3.  Number  and  Percentage  Of  Diversity  Groups  By  Class  Year 


1995 

1996 

1997 

1998 

1999 

2000 

2001 

Total  by 
group 

Female 

145 

12.8 

% 

168 

13.6 

% 

166 

14.1 

% 

190 

15.7 

% 

193 

16.7 

% 

198 

16.4 

% 

213 

18.1 

% 

1273 

15.3 

% 

Minority 

193 

17.0 

% 

197 

15.9 

% 

158 

13.4 

% 

209 

17.3 

% 

191 

16.5 

% 

172 

14.2 

% 

195 

16.6 

% 

1315 

15.8 

% 

Blue 

Chip 

154 

13.6 

% 

142 

11.5 

% 

165 

14.0 

% 

134 

11.1 

% 

138 

11.9 

% 

137 

11.3 

% 

133 

11.3 

% 

1003 

12.1 

% 

Nominat 

ion 

236 

20.8 

% 

269 

21.7 

% 

368 

31.2 

% 

281 

23.3 

% 

270 

23.4 

% 

298 

24.6 

% 

292 

24.9 

% 

2014 

24.3 

% 

Total  by 
Grad 

Year 

728 

64.1 

% 

776 

62.7 

% 

857 

72.6 

% 

814 

67.4 

% 

792 

68.6 

% 

805 

66.6 

% 

833 

71.0 

% 

5605 

67.5 

% 

Total 
Accepted 
by  Grad 
Year 

1135 

1237 

1181 

1208 

1155 

1209 

1174 

8299 

(1)  Gender.  The  trend  in  female  entrants  over  the  year  groups 
in  this  study  have  remained  relatively  stable  and  eonsistent.  Figure  6  illustrates  the 
number  of  oases  in  the  data  sample  by  gender  delineated  by  year  group.  Figure  7  oharts 
the  oandidate  multiple  mean  by  gender. 
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Figure  6,  Number  of  Cases  by  Gender  and  Graduation  Year 
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Accepted  female  applicants  to  the  Naval  Academy  have  a  slightly 
higher  candidate  multiple  mean  (differing  by  only  142  points)  but  the  difference  is  not 
significant  {t  value  =  .95,  p=  .340).  This  leads  us  to  believe  that  females  were  not  treated 
differently  in  the  admissions  process  as  they  do  not  have  any  significant  difference  in  past 
performance. 

(2)  Racial  and  Ethnic  Minority  Groups.  Figure  8  illustrates  the 
number  of  cases  in  each  respective  minority  group  in  the  data  set  by  graduating  class 
year.  The  trend  in  minority  entrants  has  remained  relatively  stable.  (The  scale  on  the 
chart  in  Figure  8,  leads  you  to  believe  that  the  cases  numbers  for  each  year  are  not  stable. 
The  cases  in  minority  groups  account  for  10%  to  18%  of  the  class  that  have 
approximately  1,200  total  cases.) 


Figure  8,  Number  of  Cases  in  Minority  Groups  by  Graduation  Year 

Figure  9  shows  that  African  American  candidates  have  a 
considerably  lower  mean  candidate  multiple  as  compared  to  other  candidates.  The  mean 
candidate  multiple  varies  from  63,401  for  whites,  to  62,869  for  the  other  minority  races. 
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60,965  for  Hispanic  and  58,307  for  African  American  candidates.  A  one  way  ANOVA 
resulted  in  an  F  value  of  240.53  (Sig.  level  of  <.001)  indicating  the  eandidate  multiple 
scores  differ  signifieantly  across  the  groups. 
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Figure  9,  Mean  Candidate  Multiple  of  Minority  Groups  by  Graduation  Year 


The  difference  in  mean  eandidate  multiple  seores  clearly  indicate  a 
gap  in  initial  qualifications  and  observed  performanee  coming  out  of  high  school  for 
African  American  ethnic  group.  This  gap  in  candidate  multiple  seores  lead  us  to  believe 
that  on  average  an  Afriean  American  candidate  may  be  treated  differently  by  the 
Admission  Board  if  they  are  to  overcome  the  candidate  multiple  gap.  This  study  will 
focus  on  the  Recommendations  of  the  Admissions  Board  (RAB)  and  how  the  RAB 
affects  performance  at  the  Naval  Academy.  The  importance  of  this  relationship  is  critieal 
to  this  study  because  we  show  that  once  candidate  multiple  is  accounted  for,  the  RAB 
beeomes  a  strong  predictor  of  success.  By  aecounting  for  raee  and  ethnicity  in  the 
analyses,  we  will  see  how  the  RAB  may  act  as  an  equalizing  factor  for  candidates  with 
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lower  candidate  multiples  as  well  as  a  performance  predictor  across  the  full  scale  of 
candidate  multiples.  Because  the  RAB  is  a  subjective  measure  awarded  to  the  candidates 
on  an  individual  basis  once  their  admissions  packages  have  been  completely  reviewed,  it 
appears  to  be  used  to  reward  unobservable  traits  that  the  Admissions  Board  finds 
important  and  critical  to  the  candidate’s  success  at  the  Naval  Academy. 

(3)  Nomination  Sources.  Peculiar  to  the  Naval  Academy 
admissions  process,  the  nomination  source  also  provides  an  opportunity  for  the 
Admissions  Board  to  reward  a  particular  group  of  applicants.  Figure  10  displays  the 
trend  in  awardees  by  nomination  source  by  class  year. 


Figure  10,  Number  of  Cases  in  Each  Nomination  Source  by  Graduation  Year 

Figure  1 1  shows  that  candidates  entering  from  active  duty  service 
as  enlisted  sailors  and  NROTC  candidates  have  a  considerably  lower  Candidate  Multiple. 
Not  only  are  enlisted/NROTC  candidates  considerably  lower,  the  highest  mean  candidate 
multiple  if  for  the  Senator  and  Congressional  nominations,  which  in  our  study  is 
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considered  a  “regular”  nomination  source.  The  means  vary  from  64,524  for  the  Senator 
and  Representative  nominees  to  58,864  for  the  enlisted  and  NROTC  candidates.  An 
ANOVA  resulted  in  an  F  value  of  399.34  (Sig.  level  of  <.00 1).  Once  again,  we  have 
identified  a  group  with  a  considerable  gap  in  observed  prior  performance  thus,  the  active 
duty  enlisted  and  NROTC  candidates  that  are  offered  an  appointment  to  the  Naval 
Academy  must  possess  unobservable  traits  that  the  Admissions  Board  awards  in  the  form 
of  the  RAB  in  the  admissions  process.  Table  4  shows  the  mean  SAT  scores  and  RAB 
points  awarded  by  nomination  source. 


Nomination  Source 


Figure  11,  Number  of  Cases  in  Each  Nomination  Source  by  Grad  Year 
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Table  4,  Mean  SAT  and  RAB  Scores  by  Nomination  Source 


N 

Mean 

SAT-M 

Mean 

SAT-V 

Mean 

RAB 

Senator  &  Representative 

3768 

671 

649 

1097 

President  &  VP 

484 

662 

637 

1533 

Quaiified  Aiternate 

2470 

656 

633 

1462 

Eniisted  -ROTC 

1285 

620 

597 

2325 

Superintendent 

189 

636 

612 

1693 

(4)  Athletic  Recruit  Status.  Recruited  athletes  are  clearly  a 
group  that  may  receive  special  treatment.  We  have  chosen  the  Blue  Chip  Athlete  to 
concentrate  on  in  this  study  because  these  athletes  are  recruited  all  over  the  country  to 
participate  in  a  specific  NCAA  DIV  I  varsity  sports  program.  For  most  schools  around 
the  country  this  is  a  sensitive  topic  for  the  Admissions  Board  as  they  try  to  balance 
advancing  the  athletic  program  with  student  performance  in  the  classroom.  The  Blue 
Chip  athlete  who  applies  to  the  Naval  Academy  must  also  possess  academic  strengths 
and  the  candidate  multiple  is  the  tool  to  highlight  the  prior  performance  of  applicants. 
The  number  of  cases  of  Blue  Chip  athletes  is  charted  by  graduation  year  in  Figure  12. 

Figure  13  shows  why  athletic  recruit  status  is  a  delicate  topic 
among  the  nation’s  Admissions  Boards.  The  non-recruits  have  a  higher  candidate 
multiple  than  recruited  athletes.  The  difference  is  4,128  {t-value  =  35.9;  P  <.001). 
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Grad  Year 

Figure  12,  Number  of  Cases  of  Blue  Chip  Athletes  by  Grad  Year 
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Figure  13,  Mean  Candidate  Multiple  by  Athletic  Recruit  Status 
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Having  a  complete  understanding  of  the  initial  eandidate  multiple 
of  the  various  speeial  groups  shows  why  the  Admission  Board  may  need  to  take  speeial 
action  to  assist  these  groups.  For  the  Admissions  Board,  the  seareh  for  diversity  and 
quality  is  where  the  subjective  RAB  adds  its  value  in  the  admissions  proeess.  By  seeking 
diversity  and  admitting  eandidates  in  groups  that  traditionally  have  a  lower  eandidate 
multiple,  the  Admission  Board  is  taking  the  ehanee  of  lowering  the  overall  quality  of  the 
eandidates  admitted,  in  terms  of  performanee,  but  also  reeommending  and  admitting 
eandidates  that  they  expeet  to  rise  to  the  ehallenge  and  perform  to  the  standards  of  the 
Naval  Aeademy  due  to  desirable  unobserved  traits  diseovered  during  the  admissions 
proeess. 

b.  Defining  Diversity  Groups  as  Independent  Variables 
This  study  divides  the  independent  variables  into  five  eategories:  basic 
demographic  data;  recruit  athletic  status;  nomination  sources;  eandidate  multiple  (CM); 
and  Reeommendations  of  the  Admissions  Board  (RAB).  Individual  variables  in  eaeh 
eategory  are  discussed  at  length  in  the  seetions  below. 

(1)  Basie  Demographie  Data.  The  demographie  data  eonsists 
of  gender  and  raee/ethnieity.  These  independent  variables  are  ineluded  in  the  regressions 
beeause  of  the  likelihood  that  gender  and  raee/ethnieity  are  treated  differently  and  eould 
be  eonsidered  for  the  award  of  additional  RAB  points  during  the  admissions  proeess. 
The  variables  are  defined  as  follows: 

(i)  Female.  This  is  a  diehotomous  variable  where  1 
represents  a  female,  and  0  otherwise. 

(ii)  African  American  (AFRAMER).  This  is  a 
diehotomous  variable  where  1  represents  the  eandidate ’s  raee  as  an  Afriean  Ameriean, 
and  0  otherwise. 

(Hi)  Hispanic  (Hispanic).  This  is  a  diehotomous  variable 
where  1  represents  the  candidate  raee  as  a  Hispanie,  and  0  otherwise. 

(iv)  Other  Minority  (Othrace).  This  is  a  diehotomous 
variable  where  1  represents  the  eandidate  raee  as  minority  other  than  Afriean  Ameriean 
or  Hispanie,  sueh  as  Asian  Ameriean,  Filipino,  Native  American,  Native 
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Hawaiian/Pacific  Islander,  Puerto  Riean,  as  well  as  if  the  eandidate  indieated  “other”  as 
their  raee  on  the  applieation  form,  otherwise  the  variable  is  0. 

(2)  Recruit  Athletic  Status.  The  athletic  status  is  also  used  as 
independent  variable  because  this  category  could  also  be  seen  as  group  that  would  benefit 
from  the  award  of  RAB  points. 

(i)  Blue  Chip  Recruited  Athlete  (Recblchp).  This  is  a 
dichotomous  variable  where  1  represents  the  candidate  was  considered  a  “blue  chip” 
recruited  athlete  by  the  Naval  Academy,  and  0  otherwise. 

(3)  Nomination  Source.  The  nomination  sources  are  also  used 
as  independent  variables  because  individual  groups  may  benefit  from  the  award  of  RAB 
points. 

(i)  Nomination  from  Senator  or  Representative 
(Nomsenrp).  This  is  a  dichotomous  variable  where  1  represents  candidate  was  nominated 
by  a  senator  or  a  representative,  and  0  otherwise. 

(ii)  Nomination  from  President  or  Vice  President 
(Nomprvp).  This  is  a  dichotomous  variable  where  1  represents  candidate  was  nominated 
by  the  President  or  Vice  President  of  the  United  States,  and  0  otherwise. 

(Hi)  Nomination  from  Secretary  of  the  Navy  (Nomenlrtc). 
This  is  a  dichotomous  variable  where  1  represents  the  candidate  was  nominated  by  the 
Secretary  of  the  Navy  from  the  enlisted  ranks  or  an  NROTC  program,  and  0  otherwise. 

(iv)  Nomination  from  the  Superintendent  (Nomsupe).  This 
is  a  dichotomous  variable  where  1  represents  candidate  was  nominated  by  the 
Superintendent  of  the  Naval  Academy,  and  0  otherwise. 

(v)  Nominated  as  a  Qualified  Alternate  (Nomqalt).  This  is 
a  dichotomous  variable  where  1  represents  candidate  was  nominated  by  USNA  as  a 
qualified  alternate,  and  0  otherwise. 

c.  Candidate  Multiple  (CM) 

The  candidate  multiple  (CM)  is  the  number  associated  with  the  statistical 
based  scoring  model  the  Naval  Academy  uses  in  the  admissions  process.  The  CM 
represents  the  high  school  performance  of  the  application  based  on  observable  data  such 
as  high  school  grade  point  average,  standardized  test  scores,  physical  aptitude  tests,  etc. 
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These  indicators  are  normalized,  computed  and  weighted  into  the  CM  algorithm.  We 
use  the  candidate  multiple  as  an  independent  variable  because  it  easily  identifies  a 
candidate’s  initial  qualification.  For  ease  of  interpretation  throughout  the  study,  we  have 
divided  the  raw  candidate  multiple  by  a  factor  of  1000.  The  raw  candidate  multiple  score 
typically  ranges  from  approximately  50,000  to  77,000. 

(1)  Candidate  Multiple/ 1000  (Cmthous).  This  is  a  continuous 
variable  representing  the  candidate  multiple  (in  thousands). 

(2)  Candidate  Multiple  Squared  (Cmthsqrd).  This  is  a 
continuous  variable  representing  the  square  of  the  candidate  multiple  (in  thousands). 
This  variable  is  used  to  specify  non-linearities  in  the  Cmthous  variable.  This  variable 
enables  the  use  of  the  quadratic  formula  in  the  regression  analysis. 

Because  the  candidate  multiple  immediately  identifies  the  initial 
quality  of  a  candidate,  we  have  separated  the  CM  variable  into  four  quartiles.  The 
variables  also  are  used  to  compute  interaction  variables  with  RAB  points. 

(3)  A  candidate  multiple  of  57,999  and  below  (CMTHL058). 
This  is  a  dichotomous  variable  with  1  representing  the  candidate  belonging  to  this 
quartile  of  candidate  multiple,  and  0  otherwise. 

(4)  A  candidate  multiple  between  60,999  and  58,000 

(CMTH5861).  This  is  a  dichotomous  variable  with  1  representing  the  candidate 
belonging  to  this  quartile  of  candidate  multiple,  and  0  otherwise. 

(5)  A  candidate  multiple  between  64,999  and  61,000 

(CMTH6165).  This  is  a  dichotomous  variable  with  1  representing  the  candidate 
belonging  to  this  quartile  of  candidate  multiple,  and  0  otherwise. 

(6)  A  candidate  multiple  above  65,000  (CMTH65HI).  This  is 
a  dichotomous  variable  with  1  representing  the  candidate  belonging  to  this  quartile  of 
candidate  multiple,  and  0  otherwise. 

d.  Recommendation  of  the  Admissions  Board  (RAB) 

The  RAB  is  the  independent  variable  in  which  this  study  is  focused.  It  is 
the  subjective  aspect  of  the  admissions  process  where  the  Admissions  Board  awards 
points  to  the  candidate.  The  sum  of  the  candidate  multiple  and  the  RAB  produces  the 
Whole  Person  Multiple  (WM).  The  Whole  Person  Multiple  if  the  final  criteria  by  which 
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a  candidate  is  considered  for  appointment  to  the  Naval  Academy.  For  ease  of 
interpretation,  we  have  divided  the  RAB  by  500  creating: 

(1)  RAB/500  (RAB500).  This  variable  is  continuous 
representing  the  numerical  amount  of  points  awarded  by  the  Admissions  Board  (divided 
by  500). 

We  have  also  created  a  few  interactive  variables  between  the  RAB 
and  the  Candidate  Multiple  to  see  if  the  interaction  effect  is  linear  or  non-linear.  The 
interaction  variables  are  discussed  in  the  remaining  sections. 

(2)  Rab500  *  CMThous  (RABCM).  This  is  a  continuous 
variable  representing  the  interaction  between  the  RAB500  (continuous)  and  CMThous 
(continuous).  The  interaction  variable  represents  the  impact  of  the  RAB  for  various 
levels  of  the  CM.  It  is  created  to  determine  whether  the  impact  of  RAB  differs  at 
different  levels  of  CM. 

Before  discussing  the  interaction  variables  we  must  explain  how 
the  ranges  of  the  candidate  multiple  were  derived.  If  we  chart  the  interaction  of  the  RAB 
(RAB500)  and  the  candidate  multiple  (CMThous)  in  a  scatter  plot,  see  Figure  14,  we  can 
clearly  see  break  points  in  the  interaction  of  these  two  variables.  What  this  figure 
displays  that  candidates  with  lower  CM  scores  tend  to  receive  larger  RABs  in  order  to  be 
considered  “qualified”  by  the  Admission  Board.  As  the  center  of  mass  line  begins  to 
change  its  slope,  at  approximately  CMThous  =  58,  it  reaches  a  point  where  the 
candidates  are  hypothesized  to  be  receiving  RABs  more  for  unobservable  traits  rather 
than  to  boost  the  CM  for  qualification.  This  range  in  the  interaction  also  experiences  an 
increase  in  scatter  plot  mass  and  the  slope  begins  to  flatten.  CMThous  =  61,  the  slope 
has  completed  its  most  drastic  changes  and  continuing  from  61  to  65  the  slope  turns 
slightly  positive  and  this  is  the  greatest  concentration  of  plots.  In  this  range,  we  find  the 
mean  of  the  CM,  so  it  is  not  surprising  that  this  is  where  most  of  the  candidates  fall  in 
terms  of  initial  observed  qualification  as  defined  by  the  CM  and  do  not  require  a  RAB  for 
an  appointment  but  are  still  receiving  RABs  based  on  desirable  traits.  From  65  to  the 
right  end  of  the  scale  we  see  that  the  slope  of  the  line  of  mass  turns  slightly  negative  and 
continues  on  that  path,  indicating  that  these  candidates  are  fully  qualified  and  may 

receive  RABs  based  on  desirable  unobserved  traits  in  their  admission  package. 
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It  must  also  be  noted  that  this  figure  elearly  illustrates  that  the 
RAB  is  awarded  throughout  the  entire  range  of  CM  and  not  just  to  a  certain  range  of  the 
CM.  It  does  clearly  indicate,  however,  that  a  candidate  must  receive  a  larger  RAB  to  be 
considered  for  appointment  when  they  have  a  lower  candidate  multiple.  But  just  because 
they  have  a  low  candidate  multiple  does  not  mean  they  are  not  “qualified.”  The 
Admissions  Board  determines  if  a  candidate  is  “qualified”  and  if  there  are  sufficient 
means  to  justify  a  large  RAB  to  a  candidate,  they  will  award  the  RAB  and  take  a 
calculated  risk  on  lower  student  performance. 

(3)  Rab500  *  CMTHL058  (RABL058).  This  is  a  continuous 
variable  representing  the  interaction  between  the  RAB500  (continuous)  and 
CMTHL058  (dichotomous).  The  interaction  variable  represents  the  impact  of  the  RAB 
to  candidates  in  the  57,999  and  below  CM  range. 

(4)  Rab500  *  CMTH5861  (RAB5861).  This  is  a  continuous 
variable  representing  the  interaction  between  the  RAB500  (continuous)  and  CMTH5861 
(dichotomous).  The  interaction  variable  represents  the  impact  of  the  RAB  to  the 
candidates  with  a  CM  between  60,999  and  58,000. 

(5)  Rab500  *  CMTH6165  (RAB6165).  This  is  a  continuous 
variable  representing  the  interaction  between  the  RAB500  (continuous)  and  CMTH6165 
(dichotomous).  The  interaction  variable  represents  the  impact  of  the  RAB  to  the 
candidates  with  a  CM  between  64,999  and  61,000. 

(6)  Rab500  *  CMTH65HI  (RAB65HI).  This  is  a  continuous 
variable  representing  the  interaction  between  the  RAB500  (continuous)  and  CMTH65HI 
(dichotomous).  The  interaction  variable  represents  the  impact  of  the  RAB  to  the 
candidates  with  a  CM  of  65,000  and  above. 

Table  5  —included  below— provides  a  summary  of  the  names  and 
definitions  of  the  explanatory  variables.  Table  6— included  below— provides  a  summary 
of  the  interaction  variables  names  and  definitions. 
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Figure  14,  Chart  of  RAB  and  Candidate  Multiple  Interaction 
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Table  5.  Explanatory  Variables  and  Definitions  Used  in  this  Study 


Explanatory  Variables 

Definitions 

Female 

Female  =  1  if  gender  is  female;  =0  if  male 

Candidate’s  Ethnieity 

Aframer  =  1  if  ethnieity  is  African  American;  =0  if  otherwise 
Hispanic  =  1  if  ethnicity  is  Hispanic;  =0  if  otherwise 

Otbrace  =  1  if  ethnicity  is  indicated  as  other  minority;  =0  if 
otherwise 

Reeruited  Blue  Chip  Athlete 

Recblcbp  =  1  if  recruited  as  a  “Blue  Chip”  athlete;  =0  if  otherwise 

Nomination  Source 

Nomsenrp  =  1  if  nomination  was  awarded  by  State  Senator  or 
Representative;  =0  if  otherwise 

Nomprvp  =  1  if  nomination  was  awarded  by  President  or  Vice 
President  of  The  United  States;  =0  if  otherwise 

Nomenrtc  =  1  if  nomination  was  awarded  by  Secretary  of  the  Navy; 
=0  if  otherwise 

Nomsupe  =  1  if  nomination  was  awarded  by  Superintendent  of 
USNA;  =0  if  otherwise 

Nomqalt  =  1  if  nomination  was  awarded  by  admission  board  as 
qualified  alternate;  =0  if  otherwise 

Candidate  Multiple/ 1000 

Cmtbous  =  continuous  variable  if  valid 

Cmtbous  =  sysmiss  if  not  valid 

Candidate  Multiple/ 1000 

Ranges 

CML058  =  1  if  Cmtbous  is  57.999  or  lower;  =0  if  otherwise 
CM5861  =  1  if  Cmtbous  is  between  58  and  60.999;  =0  if  otherwise 
CM6165  =  1  if  Cmtbous  is  between  61  and  64.999;  =0  if  otherwise 
CM65HI  =  1  if  Cmtbous  is  65  or  higher;  =0  if  otherwise 

Reeommendations  of  the 
Admission  Board  (RAB)/500 

RAB500  =  continuous  variable  if  valid 

RAB500  =  sysmiss  if  not  valid 

Table  6,  Interaction  Variables  and  Definitions  Used  in  this  Study 


Linearity  and  Interaction 
Variables 

Definitions 

Candidate  Multiple/1000  (Squared) 

Cmtbsqrd  =  continuous  variable  if  valid 
Cmtbsqrd  =  sysmiss  if  not  valid 

RAB500  *  CML058 

RABL058  =  continuous  variable  if  valid 
RABL058  =  0  if  not  valid 

RAB500  *  CM5861 

RAB5861  =  continuous  variable  if  valid 
RAB5861  =  0  if  not  valid 

RAB500  *  CM6165 

RAB6165  =  continuous  variable  if  valid 
RAB6165  =  0  if  not  valid 

RAB500  *  CM65HI 

RAB65HI  =  continuous  variable  if  valid 
RAB65HI  =  0  ifnot  valid 

40 


C.  HYPOTHESIZED  EFFECTS 

The  hypothesized  effeets  of  the  various  demographie  variables  and  the 
performanee  measures  are  a  useful  prelude  to  the  methods  of  analyses  seetion.  Table  7 
summarizes  expeeted  relationships  between  the  explanatory  variables  and  the  dependent 
variables  used  in  this  study.  For  example,  as  the  RAB500  inereases,  this  study 
hypothesizes  that  a  eandidate  is  more  likely  to  graduate  from  the  Naval  Aeademy, 
improve  in  order  of  merit  standing,  improve  eumulative  aeademie  grade  point  average, 
improve  eumulative  military  grade  point  average,  and  improve  the  likelihood  of  being 
selected  for  a  leadership  position  as  a  striper. 


Table  7.  Explanatory  Variables  and  Their  Hypothesized  Effects  on  Performance* 


Explanatory  Variables 

Hypothesized  Effect  on: 

GRAD 

PctlOOM 

CUMAQPR 

CUMMQPR 

STRIPER 

- 

+ 

- 

- 

- 

African  American 

- 

+ 

- 

- 

- 

Hispanic 

- 

+ 

- 

- 

- 

Other  Minority 

- 

+ 

- 

- 

- 

Nomination  SENRP 

+ 

- 

+ 

+ 

+ 

Nomination  PRVP 

+ 

- 

+ 

+ 

+ 

Nomination  ENRTC 

- 

+ 

- 

- 

- 

Nomination  SUPE 

- 

+ 

- 

- 

- 

Nomination  QAET 

+ 

- 

+ 

+ 

+ 

CMThous 

+ 

- 

+ 

+ 

+ 

RAB500 

+ 

- 

+ 

+ 

+ 

RABE058 

+ 

- 

+ 

+ 

+ 

RAB5861 

+ 

- 

+ 

+ 

+ 

RAB6165 

+ 

- 

+ 

+ 

+ 

RAB65HI 

+ 

- 

+ 

+ 

+ 

*  An  expected  positive  relationship  between  an  explanatory  variable  and  a 
performance  variable  is  denoted  by  a  “+”  sign,  while  a  sign  indicates  a  hypothesized 
negative  relationship.  For  example,  as  RAB500  increases,  this  study  hypothesizes  that  a 
candidate  is  more  likely  to  graduate  (+),  more  likely  improve  to  a  lower  OOM  percentile 
(-),  more  likely  to  improve  CUMAQPR  (+),  more  likely  to  improve  CUMMQPR  (+),  and 
more  likely  to  be  selected  for  a  striper  position  (+). 
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IV.  STATISTICAL  ANALYSIS  AND  RESULTS 


A.  INTRODUCTION 

The  literature  related  to  admissions  poliees  of  seleetive  colleges  identifies  women, 
racial/ethnic  minorities  and  athletes  as  being  common  challenges  for  Admissions  Offices 
that  strive  to  attract  diverse  student  body  who  are  likely  to  survive  a  rigorous  academic 
program.  At  Annapolis,  applicant  packages  are  reviewed  by  the  Admissions  Board, 
where  applicants  are  screened  to  be  considered  eligible  for  a  nomination  of  those  who 
meet  minimum  criteria  for  admissions.  A  sorting  process  begins  with  Senatorial  and 
Congressional  nominations  being  awarded  by  elected  officials  across  the  50  states  plus 
territories  of  Virgin  Islands  and  Puerto  Rico.  The  remaining  candidates  must  then  be 
ranked  and  further  screened  by  the  Admissions  Office  for  consideration  for  one  of  the 
alternate  nomination  sources,  which  include  Secretary  of  the  Navy,  Presidential  and  Vice 
Presidential  and  Qualified  Alternates.  The  ultimate  goal  is  still  to  select  applicants  with 
the  desire,  motivation,  and  ability  to  complete  the  rigorous  four-year  military  and 
academic  program  at  the  Naval  Academy. 

B,  SPECIFICATIONS  FOR  GRADUATION  MODELS 

To  identify  the  separate  impacts  of  the  quantitative  criteria  contained  in  the 
Candidate  Multiple  score  on  student  success  from  the  impact  of  qualitative  criteria 
resulting  from  subjective  decisions  of  the  Admissions  Board,  a  series  of  non-linear 
regression  models  will  be  estimated.  Step  one  estimates  the  impact  on  the  likelihood  of 
graduation  of  belonging  to  a  special  diversity  group  as  well  as  the  impact  of  receiving  an 
alternative  nomination  (compared  to  those  chosen  for  a  regular  Congressional 
Nomination.)  Information  on  quantitative  and  qualitative  scores  are  omitted  from  this 
initial  step,  which  allows  one  to  derive  initial  estimates  of  group  membership  on 
graduation. 

In  step  two,  the  Candidate  Multiple  is  added  to  the  graduation  model.  This  will 
allow  the  researcher  to  estimate  if  a  membership  in  a  diversity  group  or  alternative 
nomination  category  still  affects  the  graduation  probability,  independent  of  the  CM  score. 
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In  step  three,  the  qualitative  RAB  seore  assigned  by  the  Admissions  Board  is 
added  ot  the  model.  Again,  the  purpose  is  to  estimate  the  effect  of  demographic 
background  independent  of  the  RAB  score. 

Two  additional  graduation  models  are  specified  to  allow  for  interaction  effects 
between  the  Candidate  Multiple  (quantitative)  and  RAB  (qualitative)  scores.  In  step  four, 
the  interaction  is  introduced  by  simply  adding  a  multiplicative  term  (i.e.,  CM  *  RAB), 
whereas  in  step  five,  the  RAB  scores  are  added  to  the  model  across  four  ranges  of  the  CM 
score.  For  example,  if  someone  has  a  CM  score  of  57  (57,000  pts)  and  a  RAB  score  of  4 
(2,000  pts),  their  CM  range  would  then  be  in  the  lowest  of  four  possibilities,  or 
“RABL058.”  Thus,  if  one  “multiplies”  the  RAB  score  (4)  times  a  value  of  “1”  for  being 
in  the  lowest  CM  range  dummy  variable,  one  includes  this  RAB  score  only  in  the  first  of 
four  CM  range  interaction  variables  (i.e.,  RABL058=4  and  all  other  interaction  RAB 
variables  would  be  set  to  “0”).  This  process  allows  for  greater  flexibility  when 
estimating  RAB  impacts  on  graduation  across  differing  ranges  of  CM  scores  than  using  a 
simple  multiplicative  term  as  in  Step  four. 

1.  Findings  of  Empirical  Models 
a.  Step  One 

Table  8  shows  the  “marginal  effects”  of  the  graduate  logit  model,  the 
effect  of  a  “change  in”  each  independent  variable  on  the  probability  of  graduating.  The 
first  step  in  this  logit  model  identifies  the  groups  that  are  given  special  consideration  by 
the  Admissions  Office.  Table  4.1  shows  that  females  are  8  percentage  points  less  likely  to 
graduate  than  males,  while  African  American  and  Hispanic  applicants  are  estimated  to 
have  graduation  rates  8  points  below  that  of  whites.  These  results  suggest  a  challenge  to 
the  Admissions  Board  when  selecting  minorities  who  are  able  to  succeed  at  USNA.  This 
also  may  suggest  that  the  Naval  Academy  may  need  to  look  more  closely  at  gender-racial 
diversity  perspectives  among  the  majority  of  white  males  and  place  continued  emphasis 
on  academic  assistance,  especially  for  racial  minorities. 

We  also  notice  in  Table  8  that  Blue  Chip  athletes  are  6  points  less  likely  to 
graduate  than  those  ot  recruited  for  a  specific  Division  I  NCAA  sports  program.  This 
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also  represents  a  challenge  to  the  Admissions  Office  which  emphasizes  the  competition  at 
the  Division  I  level  and  the  indirect  value  added  to  Brigade  of  Midshipmen  of  offering 
Division  I  level  sports. 


Table  8,  Logistic  Regression  Model  of  Graduation  Marginal  Effects 


Step  1 

Step  2 

Step  3 

Step  4 

Step  5 

Gender 

-0.086** 

-0.087** 

-0.088** 

-0.087** 

-0.091** 

Race/Ethnicity 

African  American  (Aframer) 

-0.084** 

-0.062** 

-0.061** 

-0.054** 

-0.056** 

Hispanic  (Hispanic) 

-0.081** 

-0.067** 

-0.061** 

-0.058** 

-0.059** 

Other  Minority  (Othrace) 

-0.010 

-0.006 

-0.005 

-0.004 

-0.005 

Athletic  Status 

Blue  Chip  (Recblchp) 

-0.058** 

-0.036** 

-0.031* 

-0.027* 

-0.027* 

Nomination  Source 

President  &  VP  (Nomprvp) 

0.024 

0.027 

0.025 

0.022 

0.023 

Qualified  Alternate  (Nomqalt) 

0.028* 

0.034** 

0.033** 

0.032** 

0.033** 

Enlisted  &  NROTC  (Nomenrtc) 

0.004 

0.030* 

0.031* 

0.035* 

0.040** 

Superintendent  USNA  (Nomsupe) 

-0.043 

-0.040 

-0.046 

-0.035 

-0.043 

Graduation  Year 

YR96 

-0.045** 

-0.045** 

-0.043* 

-0.042* 

-0.044* 

YR97 

-0.005 

-0.008 

-0.004 

-0.004 

-0.005 

YR98 

-0.042* 

-0.044** 

-0.039* 

-0.041* 

-0.043* 

YR99 

-0.036* 

-0.045** 

-0.042* 

-0.045** 

-0.047** 

YROO 

-0.025 

-0.034 

-0.034 

-0.037* 

-0.038* 

YR01 

-0.019 

-0.029 

-0.031 

-0.036* 

-0.037* 

CM&RAB 

CM/1,000  (Cmthous) 

0.006** 

0.008** 

0.007** 

0.007** 

RAB/500  (RAB500) 

0.004** 

-0.040** 

Interactions 

CM/1000  *RAB/500 
(RABCM) 

0.001** 

Interactions  Non-linear 

(RABE058) 

(RAB5861) 

(RAB6165) 

(RAB65HI) 

0.009* 

Model  Chi-Square 

141.509 

171.226 

180.515 

189.717 

192.705 

-2  Eog  Eikelihood 

8548.64 

8518.93 

8452.3 

8443.104 

8440.116 

Pseudo-R  squared 

0.017 

0.020 

0.022 

.023 

.023 

*  Significance  Level  >  .05 
**  Significance  Level  >  .01 
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The  model  in  step  one  also  eompares  four  alternate  sources  to  those 
admitted  directly  by  Senator  or  Congressional  Nominations  (the  omitted  or  reference 
group).  Three  of  the  nomination  sources  (Presidential  &  Vice  Presidential,  Secretary  of 
the  Navy,  and  Superintendent  of  the  Naval  Academy)  have  similar  graduation  rates  as  the 
Congressional  direct  applicants,  since  their  estimated  marginal  effects  do  not  differ  from 
zero  significantly.  This  is  noteworthy  because,  as  shown  earlier,  the  average  quantitative 
criteria  contained  in  Candidate  Multiple  scores  are  generally  lower  for  those  nominated 
through  these  alternative  sources.  Appointments  awarded  through  the  Qualified 
Alternate  nomination  source,  in  spite  of  weaker  quantitative  criteria  scores,  are  found  to 
be  3  points  more  likely  to  graduate  than  those  identified  by  regular  Congressional 
nominations.  This  effect  is  statistically  significant.  One  possible  explanation  for  this 
unexpected  finding  may  be  that  the  RAB  assessments  of  the  Admissions  Board  along 
with  the  Admissions  Office’s  assignment  to  the  Qualified  Alternate  category  are 
correlated  with  an  applicant  having  a  stronger  desire  and  motivation  to  complete  the  four 
year  program  at  Annapolis. 

b.  Step  Two 

The  second  step  in  the  regression  model  adds  the  quantitative  (CM)  score 
from  the  applicant  package  to  the  graduation  model.  The  CM  score  is  based  upon  a 
formula  of  weighted  numerical  scores  representing  the  applicant’s  high  school 
performance  (i.e.,  SAT,  class  rank,  teacher  recommendations,  athletic  and  non-athletic 
extracurricular  activities).  The  addition  of  the  quantitative  information  to  the  model  does 
not  affect  the  impact  of  gender  on  graduation,  suggesting  the  distribution  of  CM  for 
females  is  similar  to  males.  We  see,  however,  nearly  a  2  point  reduction  in  the  estimated 
impact  of  diversity  group  membership  on  graduation  once  we  control  for  CM  scores  (e.g., 
African  American  CM  mean  is  58k  compared  to  whites  with  a  mean  CM  of  63k).  In 
addition,  the  impact  on  graduation  of  being  a  blue  chip  athlete,  given  CM  scores,  falls  by 
2  points.  Thus,  once  we  control  for  CM  score,  the  direct  effect  of  minority  status  falls  by 
about  25%. 

Once  we  control  for  observed  CM  scores  in  the  model  we  also  find 
interesting  changes  in  the  estimated  impact  of  applicants  being  sorted  into  one  of  the 
alternative  nomination  sources.  For  example,  the  impact  on  graduation  of  being  chosen 
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as  a  Qualified  Alternate  nomination  souree  inereases  from  2.8%  to  3.4%.  More 
importantly,  we  now  find  being  assigned  to  an  enlisted-NROTC  nomination  souree  raises 
the  probability  of  graduating  by  3  points  eompared  to  no  differenee  in  expeeted 
graduation  in  first  model  version.  This  finding  supports  the  belief  that  having  prior 
enlisted  experienee  is  signifieantly  related  to  eompletion  of  the  intense  four-year  program 
at  USNA. 

We  also  find  the  CM  score  is  positive  and  significantly  related  to 
graduation.  For  example,  an  additional  4000  points  (Mean  CM  =63,000)  is  estimated  to 
increase  the  probability  of  graduation  by  2.4  points.  This  supports  the  notion  that 
information  contained  in  the  application  package  is  positively  and  significantly  related  to 
the  ability  and  personal  drive  to  graduate  from  USNA. 

c.  Step  Three 

The  third  step  includes  the  RAB  score  (i.e.,  qualitative  criteria)  assigned  to 
applicant  packages  by  the  Admissions  Board.  The  Admissions  Board  personally  reviews 
each  complete  package  knowing  the  initial  quantitative  score  (CM)  and  adds  “RAB” 
points  (in  blocks  of  500  points  to  CM)  within  a  committee  voting  process.  Including 
qualitative  information  in  the  model  does  not  have  a  large  affect  on  the  coefficient  of 
gender  but  results  in  a  slight  decrease  in  the  effect  of  racial/ethnic  minority  on 
graduation,  suggesting  the  added  value  of  qualitative  information  positively  affects  the 
probability  of  graduation.  Once  both  quantitative  and  qualitative  information  are 
included  in  step  three,  it  is  estimated  that  females  are  8.8  percentage  points  less  likely  to 
graduate,  while  African  American  applicants  are  6.1  points  less  likely  to  graduate 
(compared  to  a  6.2  point  difference  with  just  quantitative  CM  scores  in  step  two),  and 
Hispanics  are  6.1  points  less  likely  to  graduate  (compared  to  6.7  points  less  likely  to 
graduate  in  step  two).  The  estimated  impact  of  additional  nomination  sources  is  the  same 
as  in  Step  two  of  the  model. 

In  general,  when  the  RAB  is  added  to  the  model  two  things  happen:  first, 
the  estimated  impact  of  CM  on  graduation  increases  in  size.  For  example,  each  +1000 
points  of  CM  increases  the  likelihood  of  graduation  by  0.8%,  an  increase  of  0.2%  by 
including  RAB  in  the  model.  Second,  the  RAB  coefficient  is  positive  and  significant  and 

indicates  each  500  point  RAB  results  in  a  0.4%  increased  probability  to  graduate.  For 
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example,  3  RABs  equaling  1500  points  increases  the  estimated  likelihood  of  graduation 
by  1.2%.  These  findings  suggest  that  there  is  value  added  from  utilizing  the  qualitative 
information  in  the  candidate  packages  and  the  existence  of  an  interaction  between 
qualitative  scores  (RAB)  and  quantitative  scores  (CM).  This  is  evident  because  the 
estimated  impact  of  CM  rises  significantly  when  RAB  scores  are  added  to  the  model. 

Table  9  compares  the  accuracy  of  model  predictions  in  terms  of  being  able 
to  classify  applicants  into  graduates.  Considering  that  four  out  of  five  accepted 
applicants  graduate,  the  models  of  classification  find  it  difficult  to  predict  those  more 
likely  to  attrite,  and  the  change  in  the  classification  of  attrite  may  be  a  better  indicator  of 
model  accuracy  than  total  prediction  figures.  With  this  in  mind,  we  see  below  that  the 
addition  of  quantitative  and  qualitative  criteria  improves  the  fit  of  the  graduation  model. 
Not  only  does  the  “model  Chi-Square”  increase,  referring  back  to  Table  8,  from  141.5  to 
192.7  (nearly  30%),  but  the  percentage  of  cases  that  are  correctly  predicted  to  attrite 
increases  from  46%  to  54%. 


Table  9.  Accuracy  of  Model  Predictions  of  Graduate  Model  (Attrition  Cases) 


#  of  Cases  Correctly 
Classified 

Step  1 

Step  2 

Step  3 

Step  4 

Step  5 

Attrite 

45.7 

51.4 

53.3 

54.2 

54.4 

Graduate 

68.2 

63.9 

63.0 

62.3 

62.4 

Total 

63.3 

61.2 

60.9 

60.5 

60.6 

d.  Step  Four  and  Step  Five 

The  final  two  steps  in  the  logit  model  introduce  the  interactions  of  the 
qualitative  and  the  quantitative  information.  We  model  two  interactions.  The  first  is  the 
interaction  of  RAB  and  CM  (Step  4).  The  second  divides  the  CM  score  into  four  ranges 
and  interacts  each  range  with  the  RAB  score  (Step  5). 

In  step  4,  the  estimated  coefficients  on  gender,  minority,  blue  chip  and 
nomination  source  are  similar  and  not  affected  significantly  compared  to  Step  3.  The 
interaction  effect  on  the  resulting  graduation  probability  estimated  by  step  4  is  shown  in 
Table  10  and  Figure  15.  We  notice  the  non-linear  interaction  in  the  figure,  as  the 
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increment  in  graduation  rates  inereases  at  a  deereasing  rate  for  any  level  of  RAB  along 
given  CM  ranges.  In  addition,  the  impaet  of  inerements  in  the  quantitative  RAB  scores 
for  a  given  CM  level  gradually  beeomes  greater  over  higher  levels  of  the  CM  seore.  For 
example,  at  a  57,000  CM  seore,  four  additional  RAB  points  is  estimated  to  raise  the 
likelihood  of  graduation  by  2.9%  points  (.756-. 727),  whereas  the  same  inerement  of 
RABs  at  the  66,000  CM  level  is  estimated  to  raise  the  projeeted  graduation  rate  by  4.9% 
points.  Clearly  the  impaet  of  RABs  on  graduation  differs  aeross  CM  ranges. 


Table  10,  Percent  Graduation  Estimated  by  Step  4 


Candidate  Multiple  (lOOO’s) 

RAB  points  awarded 

57 

60 

63 

66 

0  pts 

0.727 

0.751 

0.774 

0.795 

1000  pts 

0.742 

0.771 

0.797 

0.821 

2000  pts 

0.756 

0.789 

0.818 

0.844 

3000  pts 

0.770 

0.806 

0.837 

0.865 

Figure  15,  Percent  Graduation  Estimated  by  Step  4 
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Table  11  and  Figure  16  represent  the  estimated  probability  results  from 
Step  5  of  our  graduation  model  with  given  CM  and  RAB  scores.  Step  five  utilizes  the 
RAB*CM  interaction  but  breaks  the  CM  into  four  ranges.  This  adds  flexibility  to  the 
non-linear  restrictions  that  are  inherent  in  the  logit  model.  The  results  suggest  RAB 
points  awarded  to  applicants  with  a  CM<58k  have  no  impact  on  desire,  motivation  or 
ability  to  graduate.  The  largest  impact  of  RABs  on  graduation  is  found  in  the  middle 
range  of  the  CM,  around  the  CM  mean  of  63k.  For  example,  an  additional  4,000  RAB 
points  awarded  to  those  having  a  63,000  CM  score  is  expected  to  increase  the  probability 
of  graduating  by  5.3%  points  (.822-. 769),  but  only  by  3.2%  points  for  those  with  a  CM 
score  of  66,000.  This  finding  affects  28.4%  (2359  of  the  8299  cases)  of  the  accepted 
applicants  who  have  a  CM  near  the  mean  value  of  63k.  For  the  CM  ranges  either  side  of 
the  mean  CM  value  of  63k  (which  are:  58k  to  61k  and  65k+),  the  impact  is  small,  but  still 
positive. 

Figure  16  also  charts  the  resulting  graduation  probability  given  the  same 
Candidate  Multiple  and  number  of  RAB’ s.  Notice  the  lines  representing  the  increasing 
CM  values  are  no  longer  restricted  by  linear  properties.  We  now  can  visually  detect  the 
flexibility  this  model  gives  us  as  we  interpret  the  estimated  results.  The  most  dramatic 
difference  is  the  visible  peak  at  the  Candidate  Multiple  mean  value  (63k).  This  result 
suggests  that  an  entrant  who  applies  with  a  63k  Candidate  Multiple  whom  the 
Admissions  Board  finds  deserves  RAB  points  for  demonstrated  character  and 
performance  traits  will  be  more  likely  to  graduate  from  the  rigorous  4  year  program. 


Table  11.  Percent  Graduation  Estimated  by  Step  5 


Candidate  Multiple  (10 

OO’s) 

RAB  points  awarded 

57 

60 

63 

66 

(GRAB)  Opts 

0.719 

0.745 

0.769 

0.792 

(2  RABs)  1000  pts 

0.724 

0.760 

0.797 

0.809 

(4  RABs)  2000  pts 

0.729 

0.775 

0.822 

0.824 

(6  RABs)  3000  pts 

0.734 

0.789 

0.845 

0.824 
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Figure  16,  Percent  Graduation  Estimated  by  Step  5 

C.  SPECIFICATIONS  FOR  ORDER  OF  MERIT  MODELS 

Using  the  same  five  step  process  as  in  the  graduation  model,  we  estimate  the 
impact  of  both  the  quantitative  criteria  (CM)  and  the  qualitative  criteria  contained  in  the 
RAB  on  midshipmen  order  of  merit  (OOM).  We  converted  the  order  of  merit  to 
percentile  order  of  merit  by  stacking  and  ranking  the  graduate  cases  as  we  described  in 
Chapter  3.  Most  of  our  discussion  for  this  outcome  measure  is  focused  on  step  five,  as 
we  have  determined  that  step  five  is  the  most  comprehensive  and  flexible  model 
identifying  RAB  interactions  by  Candidate  Multiple  ranges. 

Table  12,  which  displays  the  “marginal  effects”  of  the  order  of  merit  model, 
reveals  that  the  coefficient  on  gender  is  insignificant.  This  finding  supports  the  belief 
that  gender  plays  no  role  in  the  overall  summary  measure  of  performance  among 
graduates.  Racial  and  ethnic  minority  groups  and  blue  chip  athletes,  on  the  other  hand. 
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are  more  likely  to  have  a  lower  order  of  merit  standings  (the  positive  sign  of  the  marginal 
effects  in  Table  12  indicate  a  higher  percentile  ranking)  upon  graduation. 


Table  12.  Marginal  Effects  of  OOM  Model 


Step  1 

Step  2 

Step  3 

Step  4 

Step  5 

Gender 

-0.914 

-0.191 

0.036 

-0.025 

-0.043 

Race/Ethnicity 

African  American  (Aframer) 

25.836** 

16.645** 

15.817** 

\6261** 

16.359** 

Hispanic  (Hispanic) 

13.351** 

7.430** 

6.732** 

6.922** 

6.958** 

Other  Minority  (Othrace) 

9.370** 

7.987** 

7.635** 

7.667** 

7.669** 

Athletic  Status 

Blue  Chip  (Recblchp) 

15.556** 

6.019** 

5.280** 

5.501** 

5.493** 

Nomination  Source 

President  &  VP  (Nomprvp) 

2.545 

0.813 

1.373 

1.256 

1.153 

Qualified  Alternate  (Nomqalt) 

2.081** 

-0.954 

-0.826 

-0.863 

-0.889 

Enlisted  &  NROTC  (Nomenrtc) 

11.302** 

0.088 

-0.610 

-0.262 

-0.366 

Superintendent  USNA  (Nomsupe) 

-0.380 

-1.142 

-1.047 

-1.170 

-1.070 

CM&RAB 

CM/1,000  (Cmthous) 

5.568** 

-2.985 

5.979** 

5.692** 

CMThous  Squared 

-0.065** 

-0.039** 

-0.069** 

-0.066** 

RAB/500  (RAB500) 

-0.692** 

3.453** 

Interactions 

CM/1000  *  RAB/500 
(RABCM) 

-0.066** 

Interactions  Non-linear 

(RABE058) 

-0.103 

(RAB5861) 

-0.654** 

(RAB6165) 

-0.558** 

(RAB65HI) 

-1.055** 

Adjusted  R  Squared 

.129 

.281 

.283 

.284 

.284 

E  value 

106.62 

229.76 

WAVMM 

197.39 

171.09 

Sig 

.000 

.000 

.000 

.000 

*  Significance  Level  >  .05 
**  Significance  Level  >  .01 


The  minority  variables  have  relatively  large  positive  coefficients,  which  fall 
somewhat  when  the  quantitative  and  qualitative  criteria  scores  are  included  in  the  order 
of  merit  performance  model.  In  Step  5  we  estimate  that  the  OOM  percentile  ranking  of 
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Hispanic  and  other  minority  groups  to  be  roughly  7%  points  higher  than  for  whites,  while 
blaeks  are  estimated  to  be  over  16%  points  higher  on  average.  With  graduating  elasses 
averaging  just  under  1,000,  these  higher  pereentiles  translate  into  minorities  being  ranked 
from  70  to  160  plaees  lower  than  whites  even  after  one  aceounts  for  their  Candidate 
Multiple  and  RAB  scores.  These  results  project  a  continued  challenge  to  the  Admissions 
Offiee  when  they  award  nominations  to  ethnic  minorities  who  are  less  likely  to  graduate 
and  to  perform  below  that  of  the  majority  white  midshipmen.  The  findings  for  Blue  Chip 
athletes  are  similar  to  those  for  minorities  and  the  large  positive  coeffieients  fall  from 
15%  points  to  5%  points  when  quantitative  and  qualitative  criteria  are  introduced  into  the 
model.  In  Step  5  blue  ehip  athletes  are  5  points  higher  in  percentile  rank  or  50  ranking 
slots  lower  than  those  that  are  not  highly  recruited  by  the  athletie  department. 

As  for  the  nomination  sources  in  this  model.  Qualified  Alternates  and 
enlisted/ROTC  nomination  sources  are  estimated  to  only  have  a  lower  class  (order  of 
merit)  standing  (higher  pereentile  number)  in  Step  One.  However,  onee  the  Candidate 
Multiple  is  aceounted  for  in  the  model  the  nomination  souree  eoeffieients  beeome 
statistically  insignificant.  Suggesting,  midshipmen  who  are  awarded  a  Congressional 
Nomination  and  graduate  perform,  on  average,  no  differently  in  overall  Order  of  Merit 
rankings  than  graduates  awarded  alternative  nomination  sources. 

Table  12  shows  that  the  quantitative  eriteria  embedded  in  the  CM  score  are 
signifieantly  related  to  summary  performance  ranking  of  aecepted  applieants  who 
graduate.  The  non-linear  impaet  of  Candidate  Multiple  seores  on  OOM  pereentile 
ranking  based  on  the  coeffieients  in  Table  12  are  caleulated  and  shown  in  Table  13.i  As 
evident  from  these  figures,  higher  Candidate  Multiple  scores  are  expeeted  to  improve  a 
graduate’s  overall  relative  class  standing  by  approximately  1.8  to  3.0  pereentage  points  or 
by  18  to  30  positions  for  eaeh  1,000  point  higher  CM  seore. 


1  The  regression  equation  of  the  eomplete  model  of  OOM  ean  be  written  as:  OOM=X+5.692*CM- 
0.066*CM^.  Thus  the  ehange  in  OOM  for  1000  pt  ehange  in  CM  is  simply  the  derivative  of  OOM  with  a 
respeet  to  the  CM,  or;  a00M=5. 692-0. 132*CM.  This  equation  is  used  to  derive  the  figures  eited  in  the 
text  above. 
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Table  13.  Estimated  Changes  in  OOM  Performance  Ranking 


+1,000  Points 
From  CM  Score 

Estimated  Change 
in  %  OOM 

57,000 

-1.83 

60,000 

-2.23 

63,000 

-2.62 

66,000 

-3.02 

The  impact  of  the  qualitative  RAB  scores  on  OOM  of  graduates  is  significant 
only  for  those  with  scores  above  58,000  and  the  relative  size  of  these  impacts  is  slightly 
smaller  compared  to  the  CM  scores.  Table  14  shows  the  relative  change  in  estimated 
OOM  percentile  for  a  given  CM  score  while  increasing  the  RAB  awards  by  1000  point 
increments.  If  we  evaluate  this  impact  for  those  with  high  Candidate  Multiple  scores  of 
65,000  and  above  (which  has  the  largest  estimated  RAB  impact  on  OOM  ranking),  we 
find  that  a  1,000  point  higher  RAB  score  (i.e.,  an  increment  of  two  RABs)  would  be 
estimated  to  lower  OOM  percentile  by  2.1%  points,  or  by  21  relative  positions  in  a 
graduating  class.  Figure  17  charts  the  estimated  impact  of  the  qualitative  value  of  the 
RAB  for  given  Candidate  Multiple  ranges  and  provides  a  visual  representation  of  the 
estimated  increase  in  class  (percentile)  ranking. 


Table  14,  Estimated  OOM  Percentile  by  Step  5 


Estimated  Change  in  OOM  from  CM  (1,000s): 

Change  in  RAB  from: 

57 

60 

63 

66 

(0-2)  OtolOOOpts 

-0.206* 

-1.308 

-1.116 

-2.11 

(2-4)  1000  to  2000  pts 

-0.206* 

-1.308 

-1.116 

-2.41 

(4-6)  2000  to  3000  pts 

-0.206* 

-1.308 

-1.116 

-2.11 

*  Not  statistically  significant  to  model 


In  summary,  the  information  contained  in  the  quantitative  Candidate  Multiple 
score  appears  to  have  a  slightly  greater  impact  on  the  overall  performance  of  graduates 
than  that  contained  in  the  qualitative  RAB  score.  In  addition,  the  assessment  of  the  entire 
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admissions  package  by  the  Admissions  Office  used  to  sort  applicants  into  one  of  the 
alternative  nomination  sources  suggests  that  applicants  with  lower  SAT  seores  and 
weaker  high  school  ranking  or  grades  but  with  higher  qualitative  RAB  scores  are  not  only 
more  likely  to  graduate  from  the  Academy,  but  are  expeeted  to  perform  equally  well 
compared  to  those  awarded  a  Congressional  Nomination. 


Figure  17,  Effect  of  CM  and  RAB  on  OOM  Estimated  by  Step  5 

D.  SPECIFICATIONS  FOR  CUMULATIVE  ACADEMIC  QPR  MODELS 

In  this  model  we  estimate  the  impaet  of  both  the  quantitative  and  qualitative 
criteria  on  a  student’s  eumulative  aeademie  QPR.  Onee  again,  we  report  overall  results 
but  foeus  our  diseussion  on  Step  5  of  the  model. 

To  fully  understand  the  impact  of  explanatory  variables  on  eumulative  aeademie 
QPR,  it  is  helpful  to  emphasize  the  seale  of  the  dependent  variable.  Academic  QPR  is 
calculated  on  a  0.0  to  4.0  seale  with  2.0  being  the  minimum  eutoff  for  graduation.  Since 
we  only  have  graduates  in  this  model,  the  range  of  the  AQPR  data  in  the  given  eases  is 
from  2.0  to  4.0.  Figure  18  shows  the  cumulative  distribution  of  AQPR  around  the 

median  value  of  2.85  with  three  0.05  inerements  on  either  side  illustrating  how  small 
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changes  in  the  scale  of  AQPR  will  result  in  relatively  large  changes  in  the  proportion  of 
graduates.  This  is  important  to  illustrate  because  the  discussion  of  the  impact  of  the 
Candidate  Multiple  and  RAB  on  AQPR  may  appear  at  first  to  be  relatively 
inconsequential  because  of  the  small  estimated  coefficients  found  in  the  model  results. 

60.0 
58.0 
56.0 

c  54.0 

_o 

I  52.0 
50.0 

_> 

■|  48.0 

E 

U  46.0 
44.0 
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40.0 

2.70  2.75  2.80  2.85  2.90  2.95  3.00 

AQPR 

Figure  18,  Cumulative  AQPR  Distributions  of  Graduates 

Table  15,  which  shows  the  “marginal  effects”  of  the  cumulative  aeademie  QPR 
model,  finds  that  gender  is  not  statistieally  signifieant,  supporting  the  notion  that,  all 
things  being  equal,  gender  plays  no  role  in  the  academic  performance  of  graduates. 
Racial  and  ethnic  minority  groups  along  with  blue  chip  athletes  have  significant  and 
negative  coefficients  in  this  model  suggesting  that  these  groups,  holding  both  quantitative 
and  qualitative  scores  constant,  have  a  lower  academic  QPR  upon  graduation.  The 
African  American  differential  suggests  that  this  group  graduates  on  average,  with  a  0.29 
point  lower  GPA  than  whites.  In  terms  of  the  cumulative  distribution,  this  means  that,  on 
average,  African  American  graduates  will  be  ranked  roughly  twenty  percent  lower  than 
whites,  or  by  200  positions  in  a  graduating  class  of  1000.  Likewise,  Hispanic  graduates 
are  found  to  graduate  with  an  average  0.13  point  lower  GPA  and  Other  Minority  groups 
0.11  point  lower  GPA  than  white  graduates. 
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Table  15.  Marginal  Effects  of  AQPR  Model 


Step  1 

Step  2 

Step  3 

Step  4 

Step  5 

Gender 

0.000 

-0.012 

-0.014 

-0.014 

-0.140 

Race/Ethnicity 

African  American  (Aframer) 

-0.484** 

-0.300** 

-0.294** 

-0.292** 

-0.294** 

Hispanic  (Hispanic) 

-0.267** 

-0.142** 

-0.132** 

-0.132** 

-0.132** 

Other  Minority  (Othrace) 

-0.153** 

-0.121** 

-0.115** 

-0.114** 

-0.115** 

Athletie  Status 

Blue  Chip  (Reeblchp) 

-0.305** 

-0.115** 

-0.107** 

-0.105** 

-0.105** 

Nomination  Source 

President  &  VP  (Nomprvp) 

-0.068** 

-0.033 

-0.039 

-0.040 

-0.039 

Qualified  Alternate  (Nomqalt) 

-0.039** 

0.020 

0.018 

0.018 

0.019 

Enlisted  &  NROTC  (Nomenrtc) 

-0.198** 

0.022 

0.027 

0.030 

0.032 

Superintendent  USNA 
(Nomsupe) 

-0.112 

0.029 

0.027 

0.030 

0.030 

Graduation  Year 

YR96 

-0.013 

-0.024 

-0.025 

-0.024 

-0.024 

YR97 

0.049 

0.024 

0.028 

0.026 

0.026 

YR98 

0.069 

0.050 

0.053** 

0.049 

0.049 

YR99 

0.051 

-0.025 

-0.025 

-0.032 

-0.031 

YROO 

0.053 

-0.031 

-0.036 

-0.041 

-0.041 

YROl 

0.130 

0.038 

0.029 

0.023 

0.023 

CM&RAB 

CM/1,000  (Cmthous) 

-0.106** 

-0.040 

-0.117** 

-0.110** 

CMThous  Squared 

0.001** 

0.001** 

0.001** 

0.001** 

RAB/500  (RAB500) 

0.009** 

-0.068** 

Interactions 

CM/1000  *  RAB/500 
(RABCM) 

0.001 

Interactions  Non-linear 

(RABE058) 

0.002 

(RAB5861) 

0.012** 

(RAB6165) 

0.011** 

(RAB65HI) 

0.019** 

Adjusted  R  Squared 

.139 

.301 

.303 

.304 

.304 

E  value 

77.01 

180.46 

171.85 

163.71 

148.04 

Sig 

.000 

.000 

.000 

.000 

.000 

*  Significance  Level  >  .05 
**  Signifieanee  Level  >  .01 
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Of  the  nomination  sources  in  this  model,  three  of  the  four  show  significance  in  the 
first  step  of  the  model  but  as  soon  as  CM  is  accounted  for  in  step  two,  nomination  source 
becomes  significant.  Again,  this  finding  is  important  in  that  these  applicants  who  were 
not  awarded  a  direct  Congressional  Nomination  and  having  lower  SAT  scores  and  lower 
high  school  GPA  nevertheless  performed  on  par  with  those  selected  directly  by 
Congressional  Senators  or  Representatives. 

The  non-linear  impact  of  Candidate  Multiple  scores  on  AQPR  specified  in  the 
model  (i.e.  a  quadratic  term)  from  the  coefficients  in  Table  15  and  are  calculated  and 
shown  in  Table  16.2  These  figures  show  higher  Candidate  Multiple  scores  improving  a 
graduate’s  overall  academic  QPR  to  range  from  0.004  to  0.022  points  for  each  1,000 
point  higher  CM  score  increment. 

Table  16.  Estimated  Change  in  AQPR  Performance  Based  on  CM 


+1,000  Points 
from  CM  Score 

Estimated  Change 
in  AQPR 

57,000 

60,000 

63,000 

66,000 

Continuing  with  the  results  of  step  5,  we  see  the  impact  of  the  RAB  on  the 
selected  Candidate  Multiple  ranges.  The  results  show  once  again  that  RABs  given  to 
entrants  categorized  in  the  lowest  Candidate  Multiple  range  are  not  statistically 
significant.  In  the  two  middle  ranges  (RAB5861  and  RAB6165),  however,  we  find  a 
significant  relationship  between  RAB  scores  and  academic  performance.  For  example, 
1,000  point  RAB  (i.e.  2  RABs)  awarded  to  an  applicant  with  a  63,000  CM  is  estimated  to 
graduate  with  an  AQPR  0.023  points  higher  than  an  applicant  that  did  not  receive  any 
RAB  points.  Table  17  shows  applicants  with  RABs  in  the  highest  range  of  Candidate 

2  The  regression  equation  of  the  eomplete  model  of  AQPR  ean  be  written  as:  AQPR=X  -  0.1 1*CM  + 
0.001  *CM^.  Thus  the  ehange  in  AQPR  for  1000  pt  ehange  in  CM  is  simply  the  derivative  of  AQPR  with  a 
respeet  to  the  CM,  or;  AAQPR=  -0.1 1  +  0.002*CM.  This  equation  is  used  to  derive  the  figures  eited  in  the 
text  above. 
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Multiples  (RAB65HI)  have  the  largest  impact  on  academic  performance,  estimated  with 
a  0.038  GPA  points  higher  per  1,000  point  RAB  awarded.  Figure  19  visually  depicts  the 
relationship  of  the  RAB  given  Candidate  Multiple  for  the  estimated  academic  QPR 
values. 


Table  17,  Estimated  Changes  in  AQPR  from  Step  5 


Estimated  Change  in  AQPR  from  CM  (1,000s): 

Change  in  RAB  from: 

57 

60 

63 

66 

(0-2)  OtolOOOpts 

.003 

.024 

.023 

.036 

(2-4)  1000  to  2000  pts 

.004 

.024 

.023 

.038 

(4-6)  2000  to  3000  pts 

.003 

.024 

.022 

.038 

Figure  19,  Estimated  Effect  of  CM  and  RAB  on  AQPR  by  Step  5 


In  summary,  both  quantitative  and  qualitative  admissions  scores  are  related  to 
academic  performance  of  graduates.  While  the  impact  of  higher  Candidate  Multiple 
scores  on  academic  GPA  increases  non-linearly,  it  is  interesting  to  note  that  the  estimated 
impacts  of  higher  RABs  on  GPA  is  relatively  larger.  For  example,  1,000  points  on  the 
Candidate  Multiple  score  in  the  60,000  to  62,000  range  is  estimated  to  result  in  an 
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increase  of  their  GPA  by  only  +.010,  whereas  1,000  more  RAB  points  in  this  CM  range 
is  expeeted  to  raise  GPA  by  0.024.  Similar  differenees  between  estimated  impaets  of  CM 
and  RAB  on  GPA  are  found  at  higher  ranges  of  the  Candidate  Multiple  seores. 

Onee  CM  and  RAB  scores  are  aecounted  for  those  receiving  direet  Congressional 
Nominations  are  no  more  likely  to  have  higher  GPA’s  than  those  awarded  an  alternative 
nomination.  The  aeademie  performance  differential  of  ethnic  minorities  and  blue  chip 
athletes  observed  in  admissions  applications  appear  to  be  perpetuated  in  college.  Both 
groups,  given  their  different  Candidate  Multiple  and  RAB  scores,  achieve  GPA’s 
signifieantly  below  that  of  white  applicants  not  highly  recruited  to  play  Division  I  level 
sports  at  Annapolis. 

E,  SPECIFICATIONS  FOR  CUMULATIVE  MILITARY  QPR  MODELS 

In  military  performanee  model  we  estimate  the  impact  of  both  quantitative  and 
qualitative  criteria  on  the  cumulative  military  QPR.  As  with  the  aeademie  QPR,  we  must 
realize  the  scale  in  whieh  we  are  working  with  in  this  model.  Military  QPR  is  also 
ealculated  on  a  0.0  to  4.0  scale,  and  once  again  only  including  graduates  in  the  model. 
Figure  20  shows  the  cumulative  distribution  of  MQPR  from  the  median  value  of  3.17 
with  three  0.05  inerements  on  either  side  illustrating  how  small  changes  in  the  scale  of 
MQPR  results  in  relatively  large  changes  in  the  proportion  of  graduates.  It  is  important 
to  reeognize  this  relationship  when  interpreting  the  estimated  impacts  of  Candidate 
Multiple  and  RAB  on  MQPR,  beeause  the  estimated  coefficients  revealed  in  the  model 
are  relatively  small. 
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Figure  20,  Cumulative  MQPR  Distributions  of  Graduates 

Table  18  shows  the  “marginal  effects”  for  the  model  of  cumulative  military  QPR. 
The  results  reveal  gender  as  not  being  statistically  significant.  Racial  and  ethnic  minority 
groups  have  significant  and  negative  coefficients  in  this  model  suggesting  that  these 
groups,  holding  both  quantitative  and  qualitative  scores  constant,  are  estimated  to  have  a 
lower  military  QPR  upon  graduation.  African  American  graduates  are  estimated  to  have 
a  0.087  lower  military  GPA  than  white  graduates.  Referring  back  to  the  distributive 
figure  above,  this  means  that  an  African  American  graduate,  on  average,  will  be  ranked 
approximately  eight  percentage  points  lower  than  whites  or  by  80  positions  in  a 
graduating  class  of  1000.  Similarly,  Hispanic  graduates  are  found  to  graduate  with  an 
average  0.06  point  lower  and  Other  Minority  groups  0.03  point  lower  than  white 
graduates.  Blue  chip  athletes  are  not  found  to  have  a  statistically  significant  effect  in  this 
model. 

A  fascinating  result  found  in  the  nomination  categories  reveals  the  Qualified 
Alternate  is  the  only  source  having  statistical  significance  in  the  model.  This  result  is 


61 


curious  as  it  estimates  the  Qualified  Alternate  has  a  0.023  point  higher  military  GPA 
upon  graduation  than  that  of  a  graduate  who  was  awarded  a  direct  Congressional 
Nomination.  In  addition,  the  Qualified  Alternate  is  only  slightly  affected  by  the 
introduction  of  quantitative  or  qualitative  eriteria,  as  the  coefficient  is  redueed  by  only 
0.002  points  by  step  five.  This  suggests  that  the  Qualified  Alternate  possesses  traits  and 
characteristies  that  are  desirable  to  inereased  military  performance,  that  are  unrelated  to 
quantitative  eriteria  (CM). 

Included  in  the  military  performanee  model,  is  the  cumulative  academic  QPR 
(Cumaqpr)  as  an  independent  variable,  and  the  results  show  that  aeeounting  for  the 
academic  influence  on  military  performance  is  highly  significant  throughout  all  steps  of 
this  model.  The  relatively  large  size  of  the  coefficient  (.491)  as  eompared  to  the  others  in 
the  model  further  suggests  that  the  military  QPR  performanee  is  highly  correlated  with 
academie  QPR  performanee.  More  specifieally  stated,  for  every  full  academic  GPA 
point,  military  GPA  is  estimated  to  inerease  by  almost  a  half  a  point. 
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Table  18,  Marginal  Effects  of  MQPR  Model 


Step  1 

Step  2 

Step  3 

Step  4 

Step  5 

Gender 

-0.007 

-0.006 

-0.007 

-0.007 

-0.007 

Raee/Ethnieity 

African  American  (Aframer) 

-0.087** 

-0.090** 

-0.087** 

-0.087** 

-0.087** 

Hispanic  (Hispanic) 

-0.596** 

-0.062** 

-0.057** 

-0.057** 

-0.060** 

Other  Minority  (Othraee) 

-0.374** 

-0.037** 

-0.035** 

-0.035** 

-0.035** 

Athletie  Status 

Blue  Chip  (Reeblehp) 

-0.001 

-0.006 

-0.002 

-0.001 

-0.001 

Nomination  Souree 

President  &  VP  (Nomprvp) 

0.018 

0.017 

0.014 

0.013 

0.014 

Qualified  Alternate  (Nomqalt) 

0.025** 

0.024** 

0.023** 

0.023** 

0.023** 

Enlisted  &  NROTC  (Nomenrte) 

-0.008 

-0.014 

-0.012 

-0.011 

-0.012 

Superintendent  USNA 
(Nomsupe) 

0.006 

0.005 

0.004 

0.005 

0.005 

Graduation  Year 

YR96 

-0.091** 

-0.090** 

-0.091** 

-0.090** 

-0.091** 

YR97 

-0.122** 

-0.122** 

-0.120** 

-0.120** 

-0.120** 

YR98 

-0.136** 

-0.135** 

-0.134** 

-0.135** 

-0.135** 

YR99 

-0.194** 

-0.192** 

-0.192** 

-0.194** 

-0.193** 

YROO 

-0.214** 

-0.212** 

-0.215** 

-0.216** 

-0.215** 

YROl 

-0.234** 

-0.232** 

-0.237** 

-0.238** 

-0.238** 

CUMAQPR 

0.486** 

0.492** 

0.491** 

0.490** 

0.491** 

CM&RAB 

CM/1,000  (Cmthous) 

0.002 

0.036* 

0.018 

0.025 

CMThous  Squared 

-.00003 

-.0003* 

-.0001 

-.0002 

RAB/500  (RAB500) 

0.005** 

-0.013 

Interactions 

CM/1000  *  RAB/500  (RABCM) 

0.0003 

Interactions  Non-linear 

(RABE058) 

0.003** 

(RAB5861) 

0.004* 

(RAB6165) 

0.005** 

(RAB65HI) 

0.007** 

Adjusted  R  Squared 

.577 

.577 

.578 

.578 

.578 

E  value 

605.21 

538.46 

512.19 

486.80 

442.32 

Sig 

.000 

.000 

.000 

.000 

.000 

*  Significance  Level  >  .05 
**  Signifieance  Level  >  .01 
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The  non-linear  impact  of  Candidate  Multiple  scores  on  MQPR  specified  in  the 
model  (i.e.,  quadratic  term)  from  the  coefficients  in  Table  18  are  calculated  and  shown  in 
Table  19.3  These  figures  are  surprising  as  they  do  not  support  the  expected  overall 
improvement  in  military  QPR  as  Candidate  Multiple  scores  increase.  Instead,  the 
positive  relationship  between  military  performance  and  the  CM  peaks  around  60,000 
after  which  higher  Candidate  Multiple  Scores  are  estimated  to  reduce  military 
performance  scores.  This  finding  is  powerful  as  we  look  at  the  estimated  impact  of  RAB 
on  military  QPR. 

Table  19.  Estimated  Change  in  MQPR  Performance  Based  on  CM 


+1,000  Points  from 
CM  Score 

Estimated  Change 
in  MQPR 

57,000 

0.0022* 

60,000 

0.001* 

63,000 

-0.0002* 

66,000 

-0.0014* 

*CM  is  not  statistically  significant  in  this  model 


The  results  of  step  5  emphasize  the  importance  of  the  non-linear  interaction  of  the 
Candidate  Multiple  with  the  RAB  scores  on  military  performance.  Where  the  CM  is 
estimated  to  negatively  affect  military  performance  as  it  increases,  the  RAB  continues  to 
impact  the  military  QPR  positively  throughout  all  ranges  of  CM  and  remains  statistically 
significant.  For  example,  1,000  point  RAB  (i.e.,  2  RABs)  awarded  to  an  applicant  with  a 
63,000  CM  is  estimated  to  raise  MQPR  by  0.011  points  compared  to  an  applicant  who 
did  not  receive  any  RAB  points.  Applicants  in  the  highest  CM  range  (RAB65HI)  who 
receive  between  1,000  and  2,000  RAB  points  have  the  largest  estimated  impact  on 
military  performance,  0.024  GPA  points  higher  per  1,000  point  RAB  awarded.  These 
results  in  particular  suggest  that  the  Admissions  Board  is  capturing  and  identifying  traits 
in  the  applicant  which  relate  positively  to  military  performance  and  are  aptly  awarded 
RAB  points.  The  negative  relationship  the  Candidate  Multiple  has  to  military 

3  As  noted  in  the  previous  footnote,  the  regression  equation  of  the  eomplete  model  of  MQPR  ean  be 
written  as:  MQPR=X  +  0.025*CM  -  0.0002*CM^.  Thus  the  ehange  in  MQPR  for  1000  pt  ehange  in 
CM  is  simply  the  derivative  of  MQPR  with  a  respeet  to  the  CM,  or;  AMQPR=  0.025  -  0.0004*CM.  This 
equation  is  used  to  derive  the  figures  in  the  eited  text  above. 
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performance,  on  the  other  hand,  suggests  that  the  Candidate  Multiple  may  not  be  a  good 
predictor  of  military  performance  on  a  stand  alone  basis  but  when  interacted  with  the 
RAB  across  the  CM  ranges,  the  impact  increases  with  the  increase  in  CM.  This 
relationship  is  broken  down  in  Table  20  by  illustrating  estimated  changes  in  MQPR  with 
1 ,000  point  increases  in  both  CM  and  RAB.  As  evidenced  by  the  relationships  of  the  two 
criteria  to  the  military  performance  measure,  the  qualitative  criteria  in  the  RAB  is 
capturing  strengths  of  the  applicant  that  are  most  evident  to  military  performance. 

Figure  21  shows  estimates  from  the  model  using  the  “marginal  effects”  for  the 
RAB  and  CM  values.  It  is  an  interesting  visual  representation  of  the  impact  of  the  RAB 
across  the  CM  range.  The  RAB  clearly  has  a  positive  impact  across  the  range  of 
Candidate  Multiple  scores  as  illustrated  by  this  representation  but  also  take  notice  of  the 
impact  the  RAB  has  to  the  higher  range  of  Candidate  Multiple  scores.  Following  the 
peak  estimate  of  MQPR  at  63,000  Candidate  Multiple  points,  notice  how  the  positive 
impact  of  the  RAB  is  greater  with  each  increasing  RAB  awarded. 


Table  20,  Estimated  Changes  in  MQPR  from  Step  5 


Estimated  Change  in  MQPR  from  CM  (1,000s): 

Change  in  RAB  from: 

57 

60 

63 

66 

(0-2)  OtolOOOpts 

.007 

.007 

.010 

.014 

(2-4)  1000  to  2000  pts 

.007 

.008 

.011 

.024 

(4-6)  2000  to  3000  pts 

.006 

.007 

.010 

.014 
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Figure  21,  Estimated  Effect  of  CM  and  RAB  on  MQPR  by  Step  5 

In  summary,  the  impact  of  qualitative  admissions  scores  on  military  performance 

remain  positive  and  significant,  whereas  the  quantitative  admissions  scores  became 

slightly  negative  as  CM  increases.  For  example,  1,000  points  on  the  Candidate  Multiple 

score  in  the  65,000  and  higher  range  is  estimated  to  have  no  significant  impact  on 

military  performance  (-.001  MQPR),  whereas  1,000  more  RAB  points  is  expected  to 

raise  military  QPR  by  0.014.  This  relationship  between  the  qualitative  criteria  and  the 

qualitative  criteria  leads  us  to  believe  that  the  military  performance  measure  is  closely 

related  to  the  awarding  of  a  RAB  from  the  Admissions  Board.  The  Qualified  Alternate  is 

estimated  to  graduate  with  a  military  QPR  0.023  point  higher  than  those  who  were 

awarded  a  direct  nomination  from  a  Congressional  source,  whereas  all  the  other 

nomination  sources  are  no  more  likely  to  have  higher  military  QPR’s.  The  academic 

performance  differential  of  ethnic  minorities  observed  in  admissions  applications  remains 

evident  in  the  military  performance  model  as  the  minority  groups  are  estimated  to 

graduate,  on  average,  with  lower  military  GPA’s  compared  to  the  white  graduates.  The 

blue  chip  athletes  regain  some  performance  ground  in  this  model  as  they  are  likely  to 
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graduate  with  similar  military  GPA’s  as  non  recruited  applicants  possibly  suggesting 
leadership  and  physical  courage  demonstrated  on  the  athletic  field  pays  off  for  these 
graduates  in  military  performance  measure. 


F.  SPECIFICATIONS  FOR  STRIPER  SELECTION  MODELS 

For  the  striper  selection  model  we  revert  back  to  the  logit  model  because  the 
outcome  performance  measure  is  dichotomous.  In  this  model,  we  estimate  the  impact  of 
both  quantitative  and  qualitative  criteria  on  striper  selections.  We  chose  to  include  this 
model  because  it  is  an  interesting  change  in  performance  perspectives.  To  be  selected  for 
a  striper  position  at  the  Naval  Academy,  midshipmen  must  be  selected  by  a  panel  of 
active  duty  military  officers  who  evaluate  their  daily  observed  performance.  In  short,  this 
model  provides  a  performance  measure  that  is  highly  subjective  but  takes  into  account 
quantitative  performance  of  the  midshipmen  to  select  midshipmen  for  leadership  billets 
in  the  Brigade  of  Midshipmen. 

The  results  of  Table  21  reveal  that  gender  is  not  statistically  significant  in  this 
model  along  with  African  American  and  Hispanic  minorities  once  the  Candidate  Multiple 
is  accounted  for.  “Other  Minority”  is  the  only  group  that  is  statistically  significant  and 
shows  that  this  group  is  4  percent  less  likely  to  be  selected  for  a  striper  position.  The 
results  find  that  blue  chip  athletes  are  1 1  percent  less  likely  to  be  selected  for  leadership 
billets.  This  result  is  not  surprising  as  most  blue  chip  athletes  spend  the  majority  of  their 
time  outside  of  company  area  practicing  or  competing  in  their  Division  I  level  varsity 
sport,  thus  reducing  face  time  and  observed  performance  while  in  leadership  roles  within 
their  company  areas.  The  year  groups  in  this  model  are  all  statistically  significant  and 
this  result  is  expected  because  the  comparison  year  group  of  1995  had  the  largest  number 
of  striper  billets  available,  thus  rendering  the  largest  selection  percentage  of  all  the  year 
groups.  The  inclusion  of  these  year  group  variables  helps  to  account  for  the  change  in 
percentage  of  striper  billets  in  each  class  based  on  the  number  of  available  positions  and 
the  number  of  midshipmen  in  the  class.  The  reader  is  referred  to  Chapter  3  for  further 
explanation  of  striper  billet  distribution  among  the  year  groups. 
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Table  21,  Marginal  Effects  of  Striper  Model 


Step  1 _ Step  2 _ Step  3  Step  4  Step  5 


Gender 

0.025 

0.023 

0.022 

0.022 

0.022 

Race/Ethnicity 

African  American  (Aframer) 

-0.044 

-0.011 

-0.011 

-0.007 

-0.009 

Hispanic  (Hispanic) 

-0.069** 

-0.049* 

-0.042 

-0.041 

-0.041 

Other  Minority  (Othrace) 

-0.051* 

-0.045* 

-0.042* 

-0.042* 

-0.041* 

Athletic  Status 

Blue  Chip  (Recblchp) 

-0.152** 

-0.117** 

-0.113** 

-0.111** 

-0.111** 

Nomination  Source 

President  &  VP  (Nomprvp) 

-0.010 

-0.003 

-0.006 

-0.008 

-0.007 

Qualified  Alternate  (Nomqalt) 

0.000 

0.010 

0.010 

0.010 

0.010 

Enlisted  &  NROTC  (Nomenrtc) 

-0.007 

0.031* 

0.032* 

0.036* 

0.035* 

Superintendent  USNA 
(Nomsupe) 

-0.016 

-0.012 

-0.017 

-0.014 

-0.016 

Graduation  Year 

YR96 

-0.039* 

-0.040* 

-0.039* 

-0.039* 

-0.039* 

YR97 

0.074** 

-0.075** 

-0.073** 

-0.075** 

-0.074** 

YR98 

0.055** 

-0.056** 

-0.054** 

-0.056** 

-0.054** 

YR99 

-0.046** 

-0.057** 

-0.059** 

-0.062** 

-0.060** 

YROO 

0.056** 

-0.067** 

-0.071** 

-0.074** 

-0.072** 

YROl 

0.054** 

-0.065** 

-0.072** 

-0.076** 

-0.074** 

CM&RAB 

CM/1,000  (Cmthous) 

0.008** 

0.011** 

0.010** 

0.011** 

RAB/500  (RAB500) 

0.007** 

-0.020 

Interactions 

CM/1000  *  RAB/500 
(RABCM) 

0.0004 

Interactions  Non-linear 

(RABE058) 

0.005** 

(RAB5861) 

0.008** 

(RAB6165) 

0.008* 

(RAB65HI) 

0.008* 

Model  Chi-Square 

134.38 

183.4 

199.38 

202.3 

202.5 

-2  Eog  Eikelihood 

6025.94 

5976.88 

5960.93 

5958.01 

5959.82 

Pseudo-R  squared 

.021 

.029 

.031 

.032 

.031 

*  Significance  Level  >  .05 
**  Significance  Level  >  .01 


The  Candidate  Multiple  is  highly  significant  and  has  a  significant  impact  on  the 
striper  selection  in  this  model.  For  example,  for  every  1,000  point  CM  increase  a 
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midshipman  is  1  percent  more  likely  to  be  selected  for  a  striper  position.  The  interesting 
results  come  when  looking  at  the  RAB  interaction  variables  and  how  the  estimated 
impact  is  nearly  the  same  for  all  CM  ranges  above  58,000.  This  result  suggests  that  it  is 
very  difficult  to  predict  military  leadership  from  application  packages  when  they  are 
evaluated  by  the  Admissions  Board. 


Table  22.  Accuracy  of  Model  Predictions  of  Striper  Model 


#  of  Cases  Correctly 
Classified 

Step  1 

Step  2 

Step  3 

Step  4 

Step  5 

Striper 

72.5 

67.3 

65.0 

64.7 

65.1 

Non-Striper 

40.9 

49.6 

51.9 

52.4 

52.0 

Total 

47.0 

53.0 

54.4 

54.7 

54.5 

Table  23  shows  the  estimates  from  the  model  using  the  “marginal  effects”  given 
RAB  and  CM  values.  For  example,  a  1,000  point  RAB  award  for  an  applicant  who  has  a 
63,000  CM  is  expected  to  increase  the  probability  of  being  selected  for  a  leadership 
striper  billet  by  1.4  percent  compared  to  an  applicant  who  did  not  receive  any  RAB 
points.  Figure  22  charts  the  estimated  impact  of  the  qualitative  value  of  the  RAB  given 
Candidate  Multiple  ranges. 


Table  23.  Estimated  Change  in  Striper  Selection  Percentage  from  Step  5 


Estimated  Change  in  Probabilitj 
Selection  from  CM  (1,00 

j  of  Striper 
[)s): 

Change  in  RAB  from: 

57 

60 

63 

66 

(0-2)  OtolOOOpts 

0.008 

0.013 

0.014 

0.018 

(2-4)  1000  to  2000  pts 

0.007 

0.015 

0.016 

0.020 

(4-6)  2000  to  3000  pts 

0.009 

0.016 

0.016 

0.020 
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Figure  22,  Effect  of  CM  and  RAB  on  Striper  Selection  by  Step  5 

In  summary,  the  military  leadership  measure  that  is  foeal  point  of  the  striper 
performance  measure  is  difficult  to  predict  at  the  time  of  the  Admissions  Board  as  they 
review  the  application  packages.  However,  both  quantitative  and  qualitative  admissions 
scores  are  positively  related  to  the  probability  of  selection  to  a  striper  billet.  While  the 
impact  of  higher  Candidate  Multiple  scores  on  probability  of  selection  to  a  striper 
position  increases  almost  linearly,  it  is  interesting  to  note  that  the  estimated  impact  of 
higher  RAB  scores  on  probability  of  striper  selection  is  significantly  greater.  For 
example,  1,000  points  on  the  Candidate  Multiple  score  at  the  mean  value  of  63,000 
increases  the  likelihood  of  selection  by  1  percentage  point,  where  a  1,000  point  increase 
in  RAB  score  increases  the  likelihood  by  1.4  percentage  points.  This  is  a  significant 
increase  given  that  only  18  to  20  percent  of  the  class  is  selected  for  these  striper 
positions. 
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G.  CHAPTER  SUMMARY 

The  chapter  summary  provides  us  with  an  opportunity  to  take  an  overall  look  at 
the  results  from  the  five  performance  models  and  summarize  the  overall  findings  and 
impacts  each  of  the  variables  have  to  the  models.  Although  gender  was  statistically 
significant  in  the  graduation  model  it  was  statistically  insignificant  in  the  other  four 
performance  outcome  models  suggesting  that  the  biggest  challenge  for  the  Admissions 
Board  is  identifying  and  admitting  females  with  the  drive  and  motivation  to  graduate 
from  the  rigorous  four  year  program  at  USNA.  Females  who  do  graduate,  all  else  equal, 
perform  on  par  with  the  males  in  the  other  four  performance  outcome  measures,  which 
suggests  that  the  Admissions  Board  is  identifying  and  admitting  females  with  the  ability 
to  succeed  and  perform  well. 

In  general,  minorities  are  faced  with  an  education  gap,  as  we  discussed  in  the 
literature  review,  where  even  at  USNA  they  are  shown  performing,  on  average,  lower  in 
all  but  the  Striper  Selection  model.  The  performance  models  reveal  minorities,  on 
average,  are  less  likely  to  graduate,  more  likely  to  be  lower  in  Order  of  Merit,  and  more 
likely  to  have  a  lower  academic  and  military  grade  point  average.  However,  in  the 
Striper  Selection  model,  African  American  and  Hispanic  graduates  are  just  as  likely  to  be 
selected  for  military  leadership  positions  their  First  Class  year  at  the  Naval  Academy. 
The  results  further  emphasizes  the  challenge  the  Admissions  Board  faces  in  identifying 
and  admitting  qualified  minority  candidates  who  have  the  drive  and  motivation  to 
succeed  in  the  rigorous  four  year  program  at  the  Academy.  Minorities,  however,  may  not 
perform  at  the  same  level  as  the  comparison  white  group  academically  but  the  Striper 
Selection  model  shows  that  African  American  and  Hispanic  graduates  are  attaining 
leadership  positions.  This  strengthens  the  argument  for  continued  diversity  within  the 
Brigade  of  Midshipmen  to  prepare  young  officers  of  all  races  and  ethnicities  for  combat 
leadership. 

Blue  Chip  Athletes  bring  a  different  dimension  of  diversity  to  the  Brigade  of 
Midshipmen.  These  athletes  embody  the  fighting  spirit  of  the  Brigade  as  they  compete  in 
the  Division  I  level  varsity  sports  programs  for  USNA.  Appropriately,  the  military 
performance  model  supports  this  generalization  because  in  the  military  performance 

model  the  Blue  Chip  Athlete  is  statistically  insignificant  suggesting  that  they  perform 

71 


equally  with  the  non-recruited  graduates.  This  statistieal  insignifieanee  is  a  revealing 
finding  because  the  blue  chip  athletes  perform,  on  average,  lower  in  all  the  other 
performance  models.  The  Blue  Chip  Athlete  is  less  likely  to  graduate,  more  likely  to 
rank  lower  in  Order  of  Merit,  more  likely  to  have  a  lower  academic  grade  point  average 
and  less  likely  to  be  selected  for  leadership  positions  within  the  Brigade  of  Midshipmen 
but  will  most  likely  perform  equally  in  military  grade  point  average  as  their  non-recruited 
peers. 

The  nomination  sources  provided  curious  results  as  they  varied  in  significance 
and  relationship  to  the  performance  outcome  measures.  Qualified  Alternate  and 
Enlisted/ROTC  nominations  are  statistically  significant  and  positive  in  the  graduation 
model  suggesting  that  those  applicants  selected  for  these  nominations  are  more  likely  to 
graduate  than  those  selected  by  the  primary  Congressional  nomination  source.  The  Order 
of  Merit  and  Academic  models  found  all  the  nomination  sources  to  be  statistically 
insignificant.  The  Military  QPR  model,  on  the  other  hand,  revealed  that  Qualified 
Alternates  are  more  likely  to  earn  a  slightly  higher  military  QPR  than  those  selected  for 
Congressional  Nominations  and  the  Striper  Selection  model  revealed  Enlisted/ROTC 
nominations  more  likely  to  be  selected  for  leadership  positions  within  the  Brigade. 

Table  24,  Estimated  Impacts  of  l,000pt  CM  vs,  l,000pt  RAB  at  Mean  CM 


Outcome  Measure 

Model  Results 

Graduation 

RAB  twice  the  impact 

Order  of  Merit 

CM  twice  the  impact 

Academic  QPR 

RAB  twice  the  impact 

Military  QPR 

RAB  positive  &  statistically  significant 
CM  not  statistically  significant 

Striper  Selection 

RAB  50%  greater  impact 

Table  24  summarizes  the  findings  of  the  estimated  impact  of  the  Candidate 
Multiple  and  the  RAB.  The  findings  vary  in  magnitude  and  relationship  to  the 
performance  outcome  measures  across  the  range  of  Candidate  Multiple  but  overall  the 
RAB  is  shown  providing  the  extra  value  added  to  the  admissions  process  that  is 
identifying  applicants  that  are  more  likely  to  succeed  at  the  Naval  Academy.  In 
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reviewing  the  performanee  models,  we  eompare  the  impaets  that  a  1,000  point  increase  of 
CM  has  versus  a  1 ,000  point  increase  in  RAB  has  on  each  of  the  outcome  measures  at  the 
mean  CM  value  of  63,000  points. 
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V.  SUMMARY  AND  CONCLUSIONS 


A.  SUMMARY 

The  Naval  Academy  is  challenged  with  selecting  well  rounded  candidates,  with 
diverse  backgrounds  and  who  bring  diverse  strengths  to  the  student  body  and  at  the  same 
time,  have  high  probabilities  of  success.  A  quantitative  system  is  maintained  by  the 
Naval  Academy  for  evaluating  applicants,  called  the  Candidate  Multiple  that  is  anchored 
on  proven  high  school  performance  measures,  such  as  SAT  and  high  school  GPA.  The 
Admissions  Board  then  adds  an  additional  input,  called  Recommendation  of  the 
Admissions  Board  (RAB),  which  is  subjective  in  nature  and  is  the  primary  qualitative 
input  to  the  Naval  Academy  admissions  process.  The  qualitative  RAB  input  is  added  to 
the  quantitative  Candidate  Multiple  to  calculate  the  Whole  Person  Multiple.  This  Whole 
Person  Multiple  is  the  score  used  for  the  final  decision  of  candidate  selection  and 
acceptance. 

1,  Candidate  Multiple 

This  study  focuses  primarily  on  validating  the  RAB  as  a  predictor  of  student 
success;  however,  it  is  imperative  to  understand  the  impact  of  the  Candidate  Multiple  on 
the  performance  outcomes  in  order  to  fully  appreciate  the  impact  of  the  RAB.  Because  of 
the  non-linear  properties  of  the  Candidate  Multiple,  four  examples  are  used  with  various 
CM  values  to  show  the  impact  of  the  Candidate  Multiple  in  the  performance  models. 
Table  25  summarizes  the  impact  of  a  1,000  point  increase  in  the  Candidate  Multiple  for 
each  of  the  performance  measures  evaluated  at  various  CM  values.  The  summary  shows 
the  non-linear  properties  of  the  performance  models  as  well,  where  3  out  of  the  4  models 
have  increasing  CM  impacts  that  are  statistically  significant.  The  impacts  of  the  1,000 
point  increase  in  CM  are  greater  as  the  Candidate  Multiple  increases,  suggesting  the 
greater  the  Candidate  Multiple  the  better  the  performance  outcome  is  likely  to  be.  The 
only  statistically  significant  model  where  the  1,000  point  increase  in  CM  actually 
decreases  in  impact  is  the  Graduation  model,  which  suggests  a  higher  Candidate  Multiple 
is  more  likely  to  graduate  but  the  1,000  point  increase  at  the  higher  CM  value  has  a 
smaller  positive  effect  on  the  outcome. 
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Table  25.  Summary  of  Impact  of  a  1,000  Point  CM  Change 


+1,000  CM  Points 
from  CM  Score 

Estimated 
Change  in 
GRAD 

Estimated 
Change  in 
%  OOM 

Estimated 
Change  in 
AQPR 

Estimated 
Change  in 
MQPR 

Estimated 
Change  in 
Striper 

57,000 

0.009 

-1.83 

0.004 

0.0022* 

60,000 

0.008 

-2.23 

0.010 

0.001* 

63,000 

0.008 

-2.62 

0.016 

-0.0002* 

66,000 

0.007 

-3.02 

0.022 

-0.0014* 

0.011 

*CM  not  statistically  significant  in  this  model 


More  speeifieally,  the  summary  table  shows  a  eandidate  with  a  Candidate 
Multiple  of  63,000  (mean  value  of  CM  for  year  groups  1995-2001)  is  estimated  to  have 
an  0.8%  better  probability  of  graduating,  would  have  been  26  plaees  higher  in  elass  rank, 
0.016  points  better  aeademieally  (on  a  4.0  seale),  and  0.9%  more  likely  to  be  seleeted  to  a 
striper  position  if  they  had  earned  1,000  more  points  in  their  Candidate  Multiple.  The 
Candidate  Multiple  has  a  positive  and  signifieant  impaet  on  the  probability  of  graduation 
but  the  impaet  of  a  1,000  point  inerease  beeomes  smaller  as  the  CM  inereases.  The 
higher  the  CM,  the  more  likely  the  eandidate  is  to  graduate.  Similarly,  the  higher  the 
Candidate  Multiple  the  lower  the  estimated  Order  of  Merit  pereentile.  For  instanee, 
inereasing  the  mean  CM  value  by  1,000  points  to  64,000  points,  the  eandidate  is 
estimated  to  be  2.62  pereentile  points  lower  whieh  equates  to  a  higher  elass  standing  and 
out  of  a  elass  of  1000  that  would  be  26  plaees  higher  in  the  standing.  The  same  inerease 
to  the  mean  CM  would  result  in  an  estimated  0.016  points  greater  eumulative  Aeademie 
QPR  (on  a  4.0  seale)  but  interestingly,  that  same  inerease  in  CM  would  not  impaet  the 
Military  QPR.  The  statistieal  insignifieanee  of  Candidate  Multiple  in  the  MQPR  model 
suggests  that  the  CM,  whieh  is  based  on  a  statistieal  seoring  model,  is  not  a  good 
predietor  of  Military  performanee  at  the  Naval  Aeademy.  Finally,  a  1,000  point  inerease 
from  the  mean  CM,  to  64,000  points,  would  inerease  a  eandidate’s  likelihood  of  being 
seleeted  for  a  striper  position  within  the  Brigade  by  0.9%. 
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2.  RAB 

Although,  the  Candidate  Multiple  is  the  baekbone  of  the  first  few  steps  of  the 
Admissions  eandidate  seleetion  proeess  at  the  Naval  Aeademy,  it  is  the  addition  of  the 
qualitative  measures  that  takes  the  seleetion  proeess  to  the  next  level,  whieh  involves 
ereation  of  the  whole  person  multiple.  This  study  has  shown  that  the  Candidate  Multiple 
has  a  statistieally  positive  and  larger  impaet  on  all  of  the  sueeess  measures  but  MQPR. 
However,  our  goal  in  this  study  was  to  uneover  the  intrinsie  value  of  the 
Reeommendations  of  the  Admissions  Board.  The  value  added  of  the  RAB  to  the  overall 
admissions  proeess  at  the  Naval  Aeademy  is  measured  by  ereating  models  that  inelude 
RAB  seores  in  addition  to  the  Candidate  Multiple  seores  for  alternative  performanee 
outeome  measures.  These  models  do  not  use  the  RAB  in  lieu  of  CM  seores  but  adds  the 
RAB  to  the  model  to  show  the  inereased  impaet  the  RAB  has  on  the  student  performanee 
models  given  the  CM  seore.  As  shown  by  our  findings  in  Chapter  4,  the  RAB  provides 
an  important  and  valuable  aspeet  to  the  Admissions  Board  as  it  is  highly  predietive  of 
sueeess  as  measured  by  the  performanee  outeome  measures  used  in  this  study.  Table  26 
summarizes  the  impaet  a  1,000  point  RAB  has  on  eaeh  outeome  performanee  measure  at 
sueeessively  higher  Cm  seores.  Like  CM,  the  impaets  of  RAB  on  student  performanee 
are  non-linear  in  nature.  That  is,  the  estimated  impaet  of  RAB  beeomes  larger  at  higher 
CM  seores  for  all  five  performanee  models. 

Table  26,  Summary  of  Impact  of  a  1,000  Point  RAB 


+1,000  RAB  Points 
from  CM  Score 

Estimated 
Change  in 
GRAD 

Estimated 
Change  in 
%  OOM 

Estimated 
Change  in 
AQPR 

Estimated 
Change  in 
MQPR 

Estimated 
Change  in 
Striper 

57,000 

0.005 

-0.206* 

0.003 

0.007 

0.008 

60,000 

0.015 

-1.308 

0.024 

0.007 

0.013 

63,000 

0.016 

-1.116 

0.023 

0.010 

0.014 

66,000 

0.017 

-2.110 

0.036 

0.014 

0.018 

*  Not  statistieally  signifieant  in  model. 
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It  is  also  interesting  to  note  that  the  impaets  of  additional  RAB  seores  an  student 
performanee  generally  exeeed  estimates  of  CM  seores.  For  example,  a  candidate  with  the 
mean  Candidate  Multiple  of  63,000  points  that  received  2  RABs  (or  1000  pts)  is  1.6% 
more  likely  to  graduate  than  a  candidate  with  a  Candidate  Multiple  of  63,000  points  with 
no  RABs  awarded.  This  impact  is  twice  that  of  a  1,000  point  increase  in  Candidate 
Multiple,  which  increases  the  graduation  probability  by  only  08%.  Unlike  the  graduation 
model,  the  Order  of  Merit  model  does  not  tell  a  similar  story  for  the  RAB.  A  candidate 
with  a  CM  of  63,000  points  who  receives  2  RABs  (or  1,000  points)  is  predicted  to  read 
1.16  percentage  points  higher  in  class  rank  percentile  (equating  to  roughly  11  places  in 
class  standing)  than  a  candidate  with  the  same  CM  who  did  not  receive  a  RAB.  In  this 
case  the  quantitative  CM  has  twice  the  impact  than  the  qualitative  RAB  (2.23  percentile 
to  1.116  percentile  increase,  respectively).  Continuing  with  Academic  QPR,  a  candidate 
with  CM  of  63,000  points  who  receives  2  RABs  (or  1,000  points)  is  estimated  to  have  a 
Academic  QPR  0.023  points  higher  (on  a  4.0  scale)  than  a  candidate  with  the  same  CM 
who  did  not  receive  any  RAB  points.  The  1 ,000  points  from  the  RAB  input  results  in  an 
estimated  improvement  of  0.023  points  to  the  AQPR,  whereas  a  1,000  point  CM  increase 
(with  no  RAB)  would  only  increase  AQPR  by  an  estimated  0.016. 

The  Candidate  Multiple  was  not  found  to  be  significant  for  the  Military  QPR 
model  but  the  RAB  was  found  to  be  positive  and  statistically  significant.  In  particular,  a 
candidate  with  a  CM  of  63,000  points  and  2  RABs  (or  1,000  points)  is  expected  to  have  a 
MQPR  0.010  points  higher  (on  a  4.0  scale)  than  a  candidate  with  the  same  CM  who  did 
not  receive  a  RAB.  This  finding  suggests  that  the  qualitative  input  to  the  admissions 
process  is  a  much  better  predictor  of  military  performance  than  the  quantitative  scoring 
model  of  the  Candidate  Multiple  alone.  Finally,  the  Striper  model  estimates  that  a 
candidate  with  a  63,000  point  CM  and  2  RABs  (or  1,000  points)  is  1.4%  more  likely  to 
be  selected  than  a  candidate  with  the  same  CM  and  no  RABs  awarded.  Once  again 
comparing  the  impact  of  the  1 ,000  point  increase  from  the  qualitative  and  the  qualitative 
side  of  things,  the  RAB  has  a  50%  greater  impact  on  striper  selection.  Looking  at  the 
relationships  and  the  magnitude  of  significance  of  the  RAB  and  the  Candidate  Multiple  to 
each  performance  measure  across  the  CM  value  range  provides  insight  to  the  qualities 
and  value  added  the  RAB  brings  to  the  Naval  Academy  admissions  process. 
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Overall,  the  RAB  is  providing  a  value  added  to  the  admissions  process  as 
evidenced  in  the  areas  highlighted  by  the  outcome  performance  measures  in  this  study. 
The  award  of  a  RAB  to  a  candidate  improved  the  outcome  in  every  model,  except  OOM, 
emphasizing  the  value  of  having  a  qualitative  input  to  the  admissions  process  and  the 
positive  impact  the  RAB  has  to  the  overall  process  in  selecting  a  well  rounded, 
diversified  and  capable  student  body. 

B,  CONCLUSIONS  &  RECOMMENDATIONS 

Based  on  our  results,  we  conclude  that  the  Recommendations  of  the  Admissions 
Board  (RAB)  add  value  to  the  overall  admissions  process  at  the  Naval  Academy.  The 
RAB  is  a  strong  predictor  of  student  performance  at  USNA.  The  value  added,  however, 
does  come  with  some  cost  to  the  Admissions  Board  in  the  form  of  the  time,  effort  and 
resources  devoted  by  the  board  members.  The  extensive  review  each  and  every  record 
goes  through  requires  a  committed  effort  from  all  the  Admissions  Board  members  as  they 
search  for  the  next  class  of  incoming  candidates.  The  costs,  however,  are  minimized  in 
that  the  time  spent  on  interviews  and  the  time  spent  by  civilians  and  officers  on  the 
Admissions  Board  are  defined  within  the  broad  definitions  of  job  responsibilities  and  do 
not  incur  additional  faculty  resource  costs  to  the  institution. 

An  immediate  recommendation  to  the  Admissions  Office  is  to  recognize  the 
predictive  quality  of  the  RAB  and  to  continue  to  employ  the  qualitative  review  of  the 
admissions  packages.  A  secondary  recommendation  is  to  invest  time  and  resources  to 
track  and  develop  explicit  criteria  on  which  RABs  are  awarded  to  candidates.  The 
process  of  documenting  and  developing  the  explicit  criteria  could  prove  to  be  valuable 
for  further  identifying  candidate  qualities  that  predict  student  performance.  More 
specifically,  it  could  also  be  used  to  identify  attributes  within  various  demographic 
groups  that  are  predictive  of  student  performance.  With  the  above  recommendations  in 
mind,  a  follow-on  recommendation  would  be  to  construct  a  database  to  record  the 
specific  quality  or  subjective  measures  for  which  a  RAB  is  awarded. 
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Further  study  in  this  area  could  focus  on  other  outcome  performance  measures 
using  the  same  or  similar  methods  to  evaluate  the  impact  of  the  RAB.  Or,  the  same 
measures  and  method  could  be  used  to  evaluate  a  larger  data  set  covering  more  year 
groups  from  the  Naval  Academy. 
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